Beyond Attention: Learning Spatio-Temporal Dynamics with Emergent Interpretable Topologies

2025-06-01

AI Summary

Existing graph attention networks (GATs) rely on predefined graph structures and dynamic attention mechanisms, introducing strong inductive biases, high computational overhead, and limited interpretability. To address these limitations in spatiotemporal forecasting tasks—such as traffic, energy, and meteorological prediction—this paper proposes InterGAT. It replaces GAT’s attention computation with a fully learnable, symmetric, unmasked node interaction matrix to automatically discover implicit spatial dependencies, and couples it with a GRU for temporal dynamics modeling. InterGAT is the first method to achieve topology-agnostic spatial modeling without any predefined graph structure or explicit attention computation, ensuring both efficiency and interpretability. On SZ-Taxi and Los-Loop datasets, it reduces prediction error by 21% and 6%, respectively, and cuts training time by 60–70%. The learned interaction matrices exhibit community-aware sparse structure; spectral analysis and clustering confirm their capacity to reveal functionally meaningful topologies that balance local and global coordination.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Spatio-temporal forecasting is critical in applications such as traffic prediction, energy demand modeling, and weather monitoring. While Graph Attention Networks (GATs) are popular for modeling spatial dependencies, they rely on predefined adjacency structures and dynamic attention scores, introducing inductive biases and computational overhead that can obscure interpretability. We propose InterGAT, a simplified alternative to GAT that replaces masked attention with a fully learnable, symmetric node interaction matrix, capturing latent spatial relationships without relying on fixed graph topologies. Our framework, InterGAT-GRU, which incorporates a GRU-based temporal decoder, outperforms the baseline GAT-GRU in forecasting accuracy, achieving at least a 21% improvement on the SZ-Taxi dataset and a 6% improvement on the Los-Loop dataset across all forecasting horizons (15 to 60 minutes). Additionally, we observed reduction in training time by 60-70% compared to GAT-GRU baseline. Crucially, the learned interaction matrix reveals interpretable structure: it recovers sparse, topology-aware attention patterns that align with community structure. Spectral and clustering analyses show that the model captures both localized and global dynamics, offering insights into the functional topology driving predictions. This highlights how structure learning can simultaneously support prediction, computational efficiency, and topological interpretabil-ity in dynamic graph-based domains.

Problem

• Modeling spatio-temporal dynamics without predefined adjacency structures
• Improving forecasting accuracy and reducing computational overhead
• Enhancing interpretability of learned spatial relationships in dynamic graphs

Innovation

• Learnable symmetric node interaction matrix
• GRU-based temporal decoder integration
• Interpretable sparse topology-aware patterns