Aggregate Computing and Many-Agent Reinforcement Learning: Towards a Hybrid Toolchain

The growing popularity of highly distributed IoT has highlighted the need for new methods to develop these systems effectively and at scale. Key distinguishing features of these systems include: (partial observability) each entity posses only a partial view of the environment in which it operates; (full distribution) there is no central entity that coordinates the entire system, as in traditional client-server architectures (instead, computation takes place directly on the IoT device or on some edge devices distributed throughout the system, near the IoT devices); (uncertainty) each entity/agent is influenced by its interactions with the environment and with other agents, introducing a level of stochasticity into the system. Over the years, numerous methods have been suggested to address these challenges, including: Aggregate Computing, a macro-programming paradigm, and Multi-Agent Reinforcement Learning, a machine learning paradigm. This thesis proposes the starting point for a hybrid toolchain that aims to exploit the potential of both aggregate computing and multi-agent reinforcement learning to develop systems capable of learning from experience and self-organizing in case of changes in the external environment. To attain this objective, we present ScaRLib, a framework designed to streamline the creation of these systems in simulated settings and JVM-based platforms. ScaRLib focuses on reducing the complexity of development by providing domain abstractions, integration with state-of-the-art tools for multiple subcomponents, a modular and extensible architecture, and a domain-specific language (DSL) to facilitate the configuration of diverse experiments. Finally, two experiments are also presented to validate the framework functionalities by testing it in basic contexts specific to this domain. These experiments were beneficial in verifying the proper functioning of the tool and highlighting its strengths, as well as identifying areas for future work.