Accurate and Efficient Multivariate Time Series Forecasting via Offline Clustering

2025-05-09

AI Summary

To address the O(L²) computational complexity of Transformer-based models in multivariate time series (MTS) forecasting—arising from global self-attention—this paper proposes an efficient时段-prototype-based modeling framework. The core method extracts high-level semantic temporal prototypes offline via K-means clustering over historical segments, thereby reducing online attention computation from pairwise token interactions to segment-to-prototype interactions, lowering complexity to O(L). A lightweight dynamic adaptation mechanism is further introduced to preserve modeling fidelity. Evaluated on multiple benchmark MTS datasets, the approach achieves state-of-the-art forecasting accuracy while accelerating inference by 3.2–5.8× and reducing GPU memory consumption by 67%. This work thus bridges the longstanding trade-off between long-range dependency capture and computational efficiency in MTS forecasting.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Accurate and efficient multivariate time series (MTS) forecasting is essential for applications such as traffic management and weather prediction, which depend on capturing long-range temporal dependencies and interactions between entities. Existing methods, particularly those based on Transformer architectures, compute pairwise dependencies across all time steps, leading to a computational complexity that scales quadratically with the length of the input. To overcome these challenges, we introduce the Forecaster with Offline Clustering Using Segments (FOCUS), a novel approach to MTS forecasting that simplifies long-range dependency modeling through the use of prototypes extracted via offline clustering. These prototypes encapsulate high-level events in the real-world system underlying the data, summarizing the key characteristics of similar time segments. In the online phase, FOCUS dynamically adapts these patterns to the current input and captures dependencies between the input segment and high-level events, enabling both accurate and efficient forecasting. By identifying prototypes during the offline clustering phase, FOCUS reduces the computational complexity of modeling long-range dependencies in the online phase to linear scaling. Extensive experiments across diverse benchmarks demonstrate that FOCUS achieves state-of-the-art accuracy while significantly reducing computational costs.

Problem

• Reducing computational complexity in multivariate time series forecasting
• Capturing long-range dependencies efficiently using offline clustering
• Improving accuracy and efficiency in traffic and weather prediction

Innovation

• Offline clustering extracts prototypes for efficiency
• FOCUS dynamically adapts patterns for accurate forecasting
• Linear complexity via prototypes reduces computational costs