Demystifying Topological Message-Passing with Relational Structures: A Case Study on Oversquashing in Simplicial Message-Passing

2025-06-06

International Conference on Learning Representations

AI Summary

Oversquashing—the excessive compression of neighborhood information during topological message passing—has long lacked a rigorous theoretical characterization in topological deep learning (TDL). Method: We introduce the first axiomatic, relation-structured framework that unifies message passing across graphs, simplicial complexes, and cellular complexes by modeling them as relational systems. This formalism enables precise definition and theoretical analysis of oversquashing. Contribution/Results: Our framework establishes the first axiomatic bridge between graph neural networks and TDL, enabling analyzable and controllable higher-order interactions. Through theoretical complexity analysis and experiments on simplicial neural networks, we demonstrate its effectiveness in mitigating oversquashing. Moreover, we generalize classical graph-theoretic results to higher-order topological settings, providing an interpretable foundation and principled guidance for TDL model design. The framework thus advances both the theoretical understanding and practical development of topological representation learning.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Topological deep learning (TDL) has emerged as a powerful tool for modeling higher-order interactions in relational data. However, phenomena such as oversquashing in topological message-passing remain understudied and lack theoretical analysis. We propose a unifying axiomatic framework that bridges graph and topological message-passing by viewing simplicial and cellular complexes and their message-passing schemes through the lens of relational structures. This approach extends graph-theoretic results and algorithms to higher-order structures, facilitating the analysis and mitigation of oversquashing in topological message-passing networks. Through theoretical analysis and empirical studies on simplicial networks, we demonstrate the potential of this framework to advance TDL.

Problem

• Analyzing oversquashing in topological message-passing networks
• Extending graph-theoretic results to higher-order structures
• Developing a unifying framework for simplicial message-passing

Innovation

• Unifying axiomatic framework for message-passing
• Extends graph results to higher-order structures
• Analyzes and mitigates oversquashing in networks