Fighter: Unveiling the Graph Convolutional Nature of Transformers in Time Series Modeling

2025-10-19

AI Summary

Transformer encoders and graph convolutional networks (GCNs) have been empirically applied to time series modeling, yet their theoretical relationship remains unclear. Method: This paper establishes a rigorous theoretical equivalence between Transformer encoders and multi-hop GCNs with dynamic adjacency matrices, showing that self-attention is mathematically equivalent to graph convolution over a time-varying graph topology. Building on this insight, we propose Fighter—a streamlined architecture that eliminates redundant linear projections, explicitly constructs dynamic adjacency matrices from attention weights, and models multi-scale temporal dependencies via multi-hop graph aggregation. Contribution/Results: Fighter is the first model to formally bridge Transformers and GCNs under a unified graph-theoretic framework, significantly enhancing interpretability. It achieves state-of-the-art or highly competitive performance across multiple standard time series forecasting benchmarks, demonstrating that structural simplification and mechanistic clarity jointly improve both transparency and predictive accuracy.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Transformers have achieved remarkable success in time series modeling, yet their internal mechanisms remain opaque. This work demystifies the Transformer encoder by establishing its fundamental equivalence to a Graph Convolutional Network (GCN). We show that in the forward pass, the attention distribution matrix serves as a dynamic adjacency matrix, and its composition with subsequent transformations performs computations analogous to graph convolution. Moreover, we demonstrate that in the backward pass, the update dynamics of value and feed-forward projections mirror those of GCN parameters. Building on this unified theoretical reinterpretation, we propose extbf{Fighter} (Flexible Graph Convolutional Transformer), a streamlined architecture that removes redundant linear projections and incorporates multi-hop graph aggregation. This perspective yields an explicit and interpretable representation of temporal dependencies across different scales, naturally expressed as graph edges. Experiments on standard forecasting benchmarks confirm that Fighter achieves competitive performance while providing clearer mechanistic interpretability of its predictions.

Problem

• Reveals Transformer's equivalence to Graph Convolutional Networks in time series
• Proposes streamlined architecture removing redundant projections for interpretability
• Models temporal dependencies as explicit graph edges across different scales

Innovation

• Transformer encoder equivalent to Graph Convolutional Network
• Attention matrix acts as dynamic adjacency for convolution
• Simplified architecture removes projections and adds graph aggregation