Neighboring-based Strategies for Multi-Agent Reinforcement Learning

Multi-Agent Reinforcement Learning introduces many new challenges to the single agent scenario, such as non-stationarity, scalability, partial observability, and credit assignment. While centralized training methods help address some of these problems, mitigating partial observability and non-stationarity, and facilitating credit assignment, they are affected by scalability issues as the number of agents increases. Centralized training methods are by far the most used and studied. However, they are not always a feasible solution in real-world scenarios, especially due to the potential limitations imposed by the structure of the agent network. On the other hand, decentralized training methods received less attention, but their potential in real-case scenarios is high. This thesis investigates different neighbor-based decentralized training strategies, proving that they can represent a valid alternative to the centralized training approach. Various distributed training methods, such as Experience Sharing, NN-Averaging, and NN-Consensus, are evaluated within a custom environment and compared against the centralized training scheme, in order to assess their efficiency and scalability.