Data Dependent Convergence Guarantees for Regression Problems in Neural Networks

It has been recently demonstrated that the artificial neural networks’ (ANN) learning under gradient descent method, can be studied using neural tangent kernel (NTK). This thesis’ goal is to show how techniques related to control theory, can be applied to model and improve the hyperparameters training dynamics. Moreover, it will be proven how by using methods from linear parameter varying (LPV) theory can allow the exact representation of the learning dynamics over its whole domain. The first part of the thesis is dedicated to the modelling and analysis of the system. The modelling of simple ANNs is hereby suggested and a method to expand this approach to larger networks is proposed. After the first part, the LPV system model’s different properties are analysed using different methods. After the modelling and analysis phase, the focus will be shifted on how to improve the neural network both in terms of stability and performances. This improvement is achieved by using state feedback on the LPV system. After setting up the control architecture, controllers based on different methods, such as optimal control and robust control, are then synthesized and their performances are compared.