Anomaly Detection in Large-Scale Cloud Systems: An Industry Case and Dataset

2024-11-13

arXiv.org

AI Summary

Large-scale cloud systems lack realistic, high-dimensional, long-term benchmark datasets for anomaly detection, hindering rigorous evaluation and advancement of industrial-strength methods. Method: We introduce the first ultra-large-scale, production-derived anomaly detection dataset from IBM Cloud—comprising 39,000 time series across 117,000 timestamps—featuring multi-source cloud-native telemetry (e.g., CPU, memory, latency, distributed traces) and fine-grained annotations of complex, real-world anomaly patterns. Leveraging this dataset, we propose a hybrid anomaly modeling framework integrating unsupervised and weakly supervised learning, accompanied by reproducible baseline results and a systematic analysis of practical challenges. Contribution/Results: Our dataset fills a critical industry gap, while experiments demonstrate substantial improvements in model generalization to real cloud environments and efficiency in industrial-grade evaluation. This work establishes a foundational data resource and methodological framework for advancing reliability research in cloud systems.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Scalability of ML Systems
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

As Large-Scale Cloud Systems (LCS) become increasingly complex, effective anomaly detection is critical for ensuring system reliability and performance. However, there is a shortage of large-scale, real-world datasets available for benchmarking anomaly detection methods. To address this gap, we introduce a new high-dimensional dataset from IBM Cloud, collected over 4.5 months from the IBM Cloud Console. This dataset comprises 39,365 rows and 117,448 columns of telemetry data. Additionally, we demonstrate the application of machine learning models for anomaly detection and discuss the key challenges faced in this process. This study and the accompanying dataset provide a resource for researchers and practitioners in cloud system monitoring. It facilitates more efficient testing of anomaly detection methods in real-world data, helping to advance the development of robust solutions to maintain the health and performance of large-scale cloud infrastructures.

Problem

• Large-scale Cloud Systems
• Issue Identification
• Data Set Limitations

Innovation

• Cloud System Dataset
• Machine Learning Analysis
• Performance Optimization