Deep Reinforcement Learning in combat simulations

Recent advancements in model-based reinforcement learning have demonstrated remarkable potential across various tasks, often achieving performance comparable to or better than model-free algorithms while requiring significantly fewer environmental steps. In contrast, multi-agent reinforcement learning typically demands a large amount of samples for effective training, making it a promising area to benefit from model-based approaches. Despite this, limited research has been conducted on model-based algorithms in multi-agent settings. This thesis investigates the potential of model-based multi-agent reinforcement learning in warfare simulations, where two battalions aim to either defend or capture a target position. We utilize the DreamerV3 algorithm to command companions independently while training them on shared experience. In addition, we introduce new encoder, decoder, and action networks tailored for DreamerV3 to handle the complex observation and action spaces of the ReLeGSim simulator. Our experiments evaluate the impact of various information sources, model sizes, and table embeddings on agent performance across increasing levels of difficulty. The results show significant improvements, achieving a 74- to 98-fold increase in sample efficiency and enhanced robustness during training compared to the previously employed model-free single-agent approach.