Transformer-based Multivariate Time Series Anomaly Localization

2025-01-15

AI Summary

Addressing the challenge of unsupervised fine-grained anomaly localization in multivariate time-series data, this paper proposes a spatiotemporal three-stage localization framework—operating at the time-step, sliding-window, and segment levels. We introduce a novel dual-metric mechanism integrating Spatiotemporal Anomaly Scores (STAS) and Statistical Feature Anomaly Scores (SFAS), and, for the first time, exploit representational disparities in Transformer self-attention patterns between normal and anomalous segments to achieve precise localization. The method synergistically combines spatial-temporal statistical modeling, self-attention behavior analysis, and multi-scale statistical feature extraction. Evaluated on both real-world and synthetic benchmarks, our approach achieves significant improvements over state-of-the-art methods in F1-score and Intersection-over-Union (IoU), while markedly reducing false positive rates. This work establishes a new paradigm for high-accuracy, interpretable anomaly localization in intelligent manufacturing and cyber-physical systems (CPS).

Technology Category

• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Intelligent Robots: Localization, Mapping, and Navigation

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Security and Privacy: Large-scale security measurements

Abstract

With the growing complexity of Cyber-Physical Systems (CPS) and the integration of Internet of Things (IoT), the use of sensors for online monitoring generates large volume of multivariate time series (MTS) data. Consequently, the need for robust anomaly diagnosis in MTS is paramount to maintaining system reliability and safety. While significant advancements have been made in anomaly detection, localization remains a largely underexplored area, though crucial for intelligent decision-making. This paper introduces a novel transformer-based model for unsupervised anomaly diagnosis in MTS, with a focus on improving localization performance, through an in-depth analysis of the self-attention mechanism's learning behavior under both normal and anomalous conditions. We formulate the anomaly localization problem as a three-stage process: time-step, window, and segment-based. This leads to the development of the Space-Time Anomaly Score (STAS), a new metric inspired by the connection between transformer latent representations and space-time statistical models. STAS is designed to capture individual anomaly behaviors and inter-series dependencies, delivering enhanced localization performance. Additionally, the Statistical Feature Anomaly Score (SFAS) complements STAS by analyzing statistical features around anomalies, with their combination helping to reduce false alarms. Experiments on real world and synthetic datasets illustrate the model's superiority over state-of-the-art methods in both detection and localization tasks.

Problem

• Anomaly Detection
• Time Series Data
• Complex Systems

Innovation

• Transformer Model
• Anomaly Detection
• Temporal-Spatial Analysis