Workload-Aware Autoscaling Support to LLM Inference in Cloud-Native Environments

The growing adoption of Large Language Models (LLM) in production environments and the emerging Model-as-a-Service paradigm pose new challenges for cloud infrastructures. Unlike traditional web services, generative inference workloads exhibit highly heterogeneous request profiles, a two-phase execution model with distinct computational and memory demands, and resource constraints that general-purpose autoscaling mechanisms are not designed to handle. This thesis investigates workload-aware autoscaling strategies for LLM inference in cloud-native environments. After reviewing the foundations of cloud computing, modern orchestration platforms, and the current state of the art in large-scale model serving, we present the design and implementation of a custom autoscaling controller that integrates two complementary scaling approaches: a reactive mode based on real-time resource saturation indicators, and a proactive mode that leverages a model based on queueing theory to grant defined service level objectives. An experimental evaluation conducted on production-grade clusters confirms that the proposed strategies improve both resource efficiency and latency compliance compared to standard cloud-native autoscaling technologies.