Anomaly detection on Kubernetes ecosystem: an AI and data-driven approach

In today’s digital landscape, computing is essential for innovation and efficiency across sectors. Cloud computing has revolutionized this space by offering scalable, cost-effective, and globally accessible services, reducing IT costs and enabling rapid growth through automatic updates. The trend towards larger data centers is driven by economies of scale, virtualization, and the rise of web-based applications. However, the complexity of cloud systems makes them prone to runtime issues like hardware failures and software faults. Ensuring faultless operation and optimizing resource use is crucial, as downtime can lead to significant financial losses. Anomalies pose challenges to cloud reliability, causing performance degradation and potential system outages. Autonomic anomaly detection has become critical, enabling cloud systems to self-manage by detecting and responding to abnormal behaviors to maintain system reliability. Effective monitoring and performance data collection are key to identifying anomalies, and ensuring scalable, flexible cloud services remain reliable. Kubernetes, an open-source container orchestration platform, plays a key role in managing cloud environments by automating the deployment, scaling, and management of containerized applications. It improves resource utilization, scalability, and system reliability, making it essential for modern cloud operations. This study explores the integration of Kubernetes for anomaly detection in cloud computing to enhance reliability. Using a web-based application as a case study, various techniques were evaluated, including statistical methods, machine learning models, and deep learning approaches. Deep learning approaches achieved the highest anomaly detection accuracy (0.982 F1 score). Integrating this system within Kubernetes significantly improved uptime, reduced error rates, and sped up anomaly responses, demonstrating Kubernetes’ potential to enhance cloud system efficiency and minimize downtime.