Dealing with Long-Term Constraints in a Hybrid Learning and Optimization Method

The increasing complexity of real-world decision-making problems, where long-term reasoning capabilities are required, is driving novel research on the integration of existing approaches. With this purpose, this thesis explores the synergies between Mathematical Optimization (MO) and Machine Learning (ML), focusing on integrating Reinforcement Learning (RL) with Constrained Optimization (CO) for complex decision-making problems. Our starting point is the UNIFY framework, an hybrid method comprising an offline ML-based phase as well as an online CO-based phase. Our core contribution is the extension of this framework to handle cumulative (long-term) constraints by recasting the problem as a Constrained Markov Decision Process (CMDP) and employing Lagrangian relaxation methods for its solution. We conduct empirical evaluations on an Energy Management System (EMS) tasked with optimal power flow allocation under long-term sustainability constraints. The results reveal: (1) the critical role of virtual parameters modeling in the learning process; (2) the challenges associated with the choice of the online optimization problem; and (3) the importance of hyperparameter tuning in optimizing the framework's performance. Overall, the proposed approach can offer strategic foresight allowing for proactive preparation for future requirements, while also retaining good performances even when dealing with strict constraints.