The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle

2026-03-18

AI Summary

This work addresses the quadratic computational complexity of dot-product self-attention in Transformers when modeling long-sequence time series. To overcome this limitation, the authors propose the Phasor Transformer module, which embeds sequence states onto the unit circle manifold and integrates trainable phase shifts with a parameter-free discrete Fourier transform (DFT). This approach introduces, for the first time in large-scale architectures, a phase-native representation coupled with a deterministic global interaction mechanism, enabling efficient O(N log N) global token communication. Evaluated on multi-frequency synthetic time series forecasting tasks, the method achieves performance comparable to conventional self-attention baselines while using significantly fewer parameters and stably capturing global dynamics.

Technology Category

Machine Learning: Time-Series/Data StreamsComputer Vision: Diffusion Models for VisionNatural Language Processing: Learning & Optimization for NLP

Application Category

Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphsSecurity and Privacy: Data transparency and provenanceSearch and Retrieval-Augmented AI: Large language models for search

Abstract

Transformer models have redefined sequence learning, yet dot-product self-attention introduces a quadratic token-mixing bottleneck for long-context time-series. We introduce the \textbf{Phasor Transformer} block, a phase-native alternative representing sequence states on the unit-circle manifold $S^1$. Each block combines lightweight trainable phase-shifts with parameter-free Discrete Fourier Transform (DFT) token coupling, achieving global $\mathcal{O}(N\log N)$ mixing without explicit attention maps. Stacking these blocks defines the \textbf{Large Phasor Model (LPM)}. We validate LPM on autoregressive time-series prediction over synthetic multi-frequency benchmarks. Operating with a highly compact parameter budget, LPM learns stable global dynamics and achieves competitive forecasting behavior compared to conventional self-attention baselines. Our results establish an explicit efficiency-performance frontier, demonstrating that large-model scaling for time-series can emerge from geometry-constrained phase computation with deterministic global coupling, offering a practical path toward scalable temporal modeling in oscillatory domains.

Problem

• Transformer
• self-attention bottleneck
• long-context time-series
• quadratic complexity
• temporal modeling

Innovation

• Phasor Transformer
• unit-circle manifold
• Discrete Fourier Transform
• time-series forecasting
• efficient attention