Cayley Graph Propagation

2024-10-04

arXiv.org

AI Summary

Graph Neural Networks (GNNs) suffer from the over-squashing problem during long-range information aggregation, where excessive neighborhood compression distorts message fidelity. Existing Expander Graph Propagation (EGP) methods mitigate this via truncated Cayley graphs, but such truncation inherently degrades the graph’s expansion properties. This paper proposes the first GNN framework performing *untruncated* message propagation on the *full* Cayley graph constructed from the group SL(2,ℤₙ), thereby strictly preserving optimal expander properties and enabling truly bottleneck-free aggregation. Our approach integrates group-theoretic graph construction, breadth-first message passing, and a GNN-layer-decoupled propagation mechanism that is agnostic to input graph topology. Experiments on multiple real-world graph benchmarks demonstrate that our method matches or surpasses state-of-the-art graph rewiring techniques in accuracy, significantly outperforms EGP, and incurs lower computational overhead.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

In spite of the plethora of success stories with graph neural networks (GNNs) on modelling graph-structured data, they are notoriously vulnerable to over-squashing, whereby tasks necessitate the mixing of information between distance pairs of nodes. To address this problem, prior work suggests rewiring the graph structure to improve information flow. Alternatively, a significant body of research has dedicated itself to discovering and precomputing bottleneck-free graph structures to ameliorate over-squashing. One well regarded family of bottleneck-free graphs within the mathematical community are expander graphs, with prior work$unicode{x2014}$Expander Graph Propagation (EGP)$unicode{x2014}$proposing the use of a well-known expander graph family$unicode{x2014}$the Cayley graphs of the $mathrm{SL}(2,mathbb{Z}_n)$ special linear group$unicode{x2014}$as a computational template for GNNs. However, in EGP the computational graphs used are truncated to align with a given input graph. In this work, we show that truncation is detrimental to the coveted expansion properties. Instead, we propose CGP, a method to propagate information over a complete Cayley graph structure, thereby ensuring it is bottleneck-free to better alleviate over-squashing. Our empirical evidence across several real-world datasets not only shows that CGP recovers significant improvements as compared to EGP, but it is also akin to or outperforms computationally complex graph rewiring techniques.

Problem

• Addresses over-squashing in graph neural networks (GNNs)
• Proposes propagation over complete Cayley graphs for better expansion
• Improves information flow without truncation compared to prior methods

Innovation

• Uses complete Cayley graph structure
• Avoids truncation for better expansion
• Improves information flow in GNNs