FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting

2026-05-31

AI Summary

This work addresses the limitations of conventional Transformers in modeling multivariate time series, which struggle to capture high-frequency transient signals due to the low-pass filtering nature of self-attention. Existing frequency-domain approaches further fall short as they rely on fixed bases and time-invariant modulation, rendering them incapable of adapting to dynamic spectral variations. To overcome these issues, the authors propose FAiT, a novel architecture that incorporates an invertible attention mechanism to construct a learnable high-pass branch for recovering high-frequency components. Additionally, a Dynamic Time-Frequency Modulation (DTFM) module is introduced to adaptively recalibrate energy across frequency bands based on input-specific characteristics. This approach transcends the static constraints of traditional frequency-domain modeling, achieving state-of-the-art performance on multiple benchmark datasets while maintaining computational efficiency over existing Transformer-based and frequency-enhanced methods.

Abstract

While Transformer-based architectures have established themselves as a dominant paradigm in Multivariate Time Series Forecasting (MTSF), their core self-attention mechanism inherently functions as a low-pass filter, systematically smoothing out high-frequency signals vital for sharp local changes. Recent advancements have increasingly incorporated frequency-domain operations to address this bias, however, most existing designs rely on fixed spectral bases and apply sequence-wise (uniform) modulation, implicitly assuming a time-invariant frequency response. This overlooks a key property of real-world series that their spectral characteristics often evolve over time, making uniform modulation insufficient for capturing fine-grained temporal dynamics. To tackle these limitations, we propose FAiT, a Frequency-Aware inverted Transformer. Specifically, FAiT rectifies the spectral bias internally through Inverted Attention, which interprets the attention map as a learnable low-pass operator and constructs a dedicated complementary high-pass branch by inverting the attention matrix to recover attenuated transient signals. Furthermore, FAiT introduces Dynamic Temporal-Frequency Modulation (DTFM), which synthesizes instance-conditioned weights to adaptively re-calibrate the energy of spectral sub-bands, enabling fine-grained control over evolving multi-scale patterns. Extensive experiments on widely used benchmarks demonstrate that FAiT consistently outperforms state-of-the-art Transformer-based and frequency-enhanced baselines, while maintaining computational efficiency.

Problem

• Multivariate Time Series Forecasting
• Frequency Domain
• Spectral Bias
• Time-Varying Dynamics
• High-Frequency Signals

Innovation

• Inverted Attention
• Dynamic Temporal-Frequency Modulation
• Frequency-Aware Transformer
• High-Pass Signal Recovery
• Time-Varying Spectral Modeling