Hyperparameter Tuning for Deep Learning Models using Ray

Hyperparameter optimization is one of the most critical stages in the development of deep learning models, as it directly influences both generalization performance and training stability. From a formal perspective, this task can be interpreted as an optimization problem in a high-dimensional space, typically non-convex and characterized by noisy, expensive evaluations of the objective function. In such a setting, the use of distributed frameworks becomes essential to achieve an efficient and scalable exploration of the search space. This thesis investigates Ray Tune, the hyperparameter tuning library integrated into the Ray distributed computing framework, as a systematic solution for exploring hyperparameter configurations through a variety of search algorithms. Experimental evaluation was first conducted on the CIFAR-10 dataset using standard convolutional architectures, providing a controlled environment for comparing search algorithms and resource allocation policies. The study was then extended to a real-world task involving high-resolution medical imaging, characterized by increased architectural and computational complexity. The results indicate that schedulers like ASHA and PBT significantly reduce the computational budget required to identify promising configurations while maintaining or improving final model performance. Bayesian methods prove effective when the hyperparameter space is smooth and continuous, but show limitations in discrete or irregular domains, where repeated sampling of identical configurations may occur. Overall, the findings demonstrate that Ray Tune is a robust and versatile framework for hyperparameter optimization, capable of combining theoretical soundness, computational efficiency, and practical applicability, including scenarios characterized by substantial computational demands.