Learning-Based Control for Nano-Drones Flight: From Simulation to Reality

Recent advances in deep reinforcement learning have opened new possibilities for autonomous aerial robots, particularly nano-drones, which are among the most agile yet difficult platforms to control. Their small size and limited hardware resources make classical control design challenging, while learning-based approaches promise greater adaptability but often face difficulties when transferring from simulation to reality. This thesis explores reinforcement learning for low-level motor control on the Crazyflie 2.1 nano-quadrotor. A complete experimental pipeline was developed, combining massively parallel simulation with the Proximal Policy Optimization (PPO) algorithm, multi-seed evaluation, and deployment on the real platform using only onboard sensing. The learned controller successfully transferred to real flight, achieving stable hovering and accurate trajectory tracking. Performance was comparable to state-of-the-art solutions, confirming that key design elements such as rotor-delay modeling and action history are critical for bridging the simulation-to-reality gap. An ablation study further demonstrated that removing these elements severely compromises stability, underscoring their importance for reliable deployment. Overall, the results show that reinforcement learning can be applied effectively to nano-drones for direct motor control, providing a reproducible pipeline that bridges simulation and reality. At the same time, the study highlights current limitations in training stability and robustness, and points toward future research aimed at improving efficiency and extending learned controllers to more complex flight scenarios.