SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting

2026-09-17

AI Summary

本文针对长期多变量时间序列预测难题，提出了一种基于变压器模型SETTer的新方法，通过解耦自注意力和混合掩码技术有效捕捉短期与长期模式。

Abstract

Long-term multivariate time series plays a significant role in many application areas such as power systems, trading, etc. However, their accurate prediction is quite difficult for conventional forecasting methods as they often exhibit high dimensionality and complex relationships. Recent works show that transformer-based approaches are quite effective for long-term forecasting thanks to their attention mechanism. However, in the presence of complex high-dimensional inputs, they show evidence of oversmoothing, limited capacity, and opacity. To this end, this paper introduces SETTer, a transformer-based model that addresses these challenges by incorporating novel techniques for decoupled self-attention and hybrid masking. The proposed techniques enable SETTer to effectively capture the dominant short- and long-term patterns across the temporal and channel dimensions. In addition, we enrich the model layers with simple explainable structures that indicate the discriminative pattern of SETTer. We show that with a single-layer transformer architecture, SETTer can effectively model long-term dependencies in the presence of varying data complexities. Extensive experiments on real-word benchmark datasets for long-term multivariate time series forecasting demonstrate that SETTer outperforms state-of-the-art models in 88% of the scenarios.

Problem

• long-term multivariate time series
• high dimensionality
• complex relationships
• oversmoothing
• limited capacity

Innovation

• Sparse-Encoder Transformer
• decoupled self-attention
• hybrid masking
• long-term multivariate time series forecasting