European Space Agency Benchmark for Anomaly Detection in Satellite Telemetry

2024-06-25

arXiv.org

AI Summary

Current satellite telemetry anomaly detection suffers from a lack of interpretable and reproducible multivariate time-series benchmarks, hindering practical machine learning deployment. To address this, we introduce the first publicly available, spacecraft-oriented multivariate time-series anomaly detection benchmark, incorporating real-world, expert-annotated telemetry data from two ESA missions. We propose a dedicated benchmarking framework for spacecraft telemetry and a hierarchical evaluation methodology grounded in operational semantics—better aligned with mission control requirements. We systematically evaluate state-of-the-art supervised and unsupervised algorithms, exposing critical performance limitations under realistic conditions. All benchmark datasets, source code, and evaluation tools are fully open-sourced. This work advances methodological standardization and reproducibility in anomaly detection research and establishes foundational infrastructure for intelligent satellite operations and maintenance.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Machine learning has vast potential to improve anomaly detection in satellite telemetry which is a crucial task for spacecraft operations. This potential is currently hampered by a lack of comprehensible benchmarks for multivariate time series anomaly detection, especially for the challenging case of satellite telemetry. The European Space Agency Benchmark for Anomaly Detection in Satellite Telemetry (ESA-ADB) aims to address this challenge and establish a new standard in the domain. It is a result of close cooperation between spacecraft operations engineers from the European Space Agency (ESA) and machine learning experts. The newly introduced ESA Anomalies Dataset contains annotated real-life telemetry from three different ESA missions, out of which two are included in ESA-ADB. Results of typical anomaly detection algorithms assessed in our novel hierarchical evaluation pipeline show that new approaches are necessary to address operators' needs. All elements of ESA-ADB are publicly available to ensure its full reproducibility.

Problem

• Lack of standardized benchmarks for satellite telemetry anomaly detection
• Need improved multivariate time series anomaly detection methods
• Addressing spacecraft operators' needs through comprehensive evaluation framework

Innovation

• Developed ESA Anomaly Detection Benchmark
• Introduced hierarchical evaluation pipeline
• Provided annotated real-life telemetry datasets