Multi-Agent Reinforcement Learning of Swarm Behaviours with Graph Neural Networks: prototype and first experiments

In a distributed multi-agent system involving intelligent agents, a typical problem consists of coordinating agents to perform complex global goals. In this thesis, we consider a swarm of drones that need to enact certain swarming scenarios using a Multi-Agent Reinforcement Learning approach. In general, three different approaches are proposed for designing this kind of systems: manual design, in which developers design all the necessary algorithms to achieve the desired behavior; automatic design, which involves employing machine learning techniques to learn the correct policy to apply; and a hybrid approach that combines both. In this work we consider the automatic approach. Specifically, we use a variation of the Deep Q-Network (DQN) algorithm, which combines Q-learning with Graph Neural Networks (GNNs) to enable efficient decision-making in complex environments through Multi-Agent Reinforcement Learning. To analyze the effectiveness of this technique, we replicate some swarming scenarios and examine how accurately they are reproduced. Another important aspect of this article is testing the scalability of this technique to clarify how many agents a system can handle with this design.