Short-Range Oversquashing

2025-11-25

AI Summary

This work reveals that Message Passing Neural Networks (MPNNs) suffer from oversquashing even on short-range graph tasks—challenging the prevailing view that oversquashing affects only long-range tasks. The authors decouple oversquashing into two distinct mechanisms: (i) short-range bottlenecks, arising from local neighborhood structure-induced information capacity limits, and (ii) long-range gradient vanishing. Crucially, existing mitigation strategies—such as virtual nodes—are shown to be ineffective against short-range bottlenecks. Through theoretical analysis and empirical evaluation, the study demonstrates that Graph Transformers, leveraging global attention, effectively alleviate this bottleneck and significantly outperform MPNNs on short-range tasks. This is the first systematic characterization of short-range oversquashing as a standalone phenomenon, with an independent causal origin. The findings provide novel theoretical insights and empirical evidence to guide principled architecture selection in graph representation learning.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

Message Passing Neural Networks (MPNNs) are widely used for learning on graphs, but their ability to process long-range information is limited by the phenomenon of oversquashing. This limitation has led some researchers to advocate Graph Transformers as a better alternative, whereas others suggest that it can be mitigated within the MPNN framework, using virtual nodes or other rewiring techniques. In this work, we demonstrate that oversquashing is not limited to long-range tasks, but can also arise in short-range problems. This observation allows us to disentangle two distinct mechanisms underlying oversquashing: (1) the bottleneck phenomenon, which can arise even in low-range settings, and (2) the vanishing gradient phenomenon, which is closely associated with long-range tasks. We further show that the short-range bottleneck effect is not captured by existing explanations for oversquashing, and that adding virtual nodes does not resolve it. In contrast, transformers do succeed in such tasks, positioning them as the more compelling solution to oversquashing, compared to specialized MPNNs.

Problem

• MPNNs suffer from oversquashing in short-range graph tasks
• Existing explanations fail to capture short-range bottleneck effects
• Transformers outperform specialized MPNNs in resolving oversquashing issues

Innovation

• Identifies short-range oversquashing bottleneck in MPNNs
• Distinguishes bottleneck from vanishing gradient mechanisms
• Positions transformers as superior solution to oversquashing