Demystifying MPNNs: Message Passing as Merely Efficient Matrix Multiplication

2025-01-31

AI Summary

This work addresses the contradictory degradation in performance and lack of theoretical understanding of Graph Neural Networks (GNNs) on deep architectures, sparse graphs, and homophilous graphs. Methodologically, it introduces the first rigorous theoretical framework that formalizes *k*-layer message passing as matrix exponentiation over *k*-hop neighborhoods—unifying the characterization of information aggregation, cycle-induced interference, and structural-feature entanglement. By modeling gradient propagation, it identifies gradient degradation—not over-smoothing—as the primary bottleneck in deep GNNs, and establishes a structure-feature decoupling analytical paradigm. Contributions include: (i) the first matrix-power-based rigorous foundation for Message Passing Neural Networks (MPNNs), explaining their empirical success; and (ii) interpretable, principled guidelines for layer-depth selection, normalization design, and training on sparse graphs—substantially improving generalization and interpretability.

Technology Category

Machine Learning: Graph-based Machine LearningReasoning under Uncertainty: Graphical ModelsData Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSocial Networks and Social Media: Influence propagation, information diffusion, and the prediction on networksSemantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

While Graph Neural Networks (GNNs) have achieved remarkable success, their design largely relies on empirical intuition rather than theoretical understanding. In this paper, we present a comprehensive analysis of GNN behavior through three fundamental aspects: (1) we establish that extbf{$k$-layer} Message Passing Neural Networks efficiently aggregate extbf{$k$-hop} neighborhood information through iterative computation, (2) analyze how different loop structures influence neighborhood computation, and (3) examine behavior across structure-feature hybrid and structure-only tasks. For deeper GNNs, we demonstrate that gradient-related issues, rather than just over-smoothing, can significantly impact performance in sparse graphs. We also analyze how different normalization schemes affect model performance and how GNNs make predictions with uniform node features, providing a theoretical framework that bridges the gap between empirical success and theoretical understanding.

Problem

• Graph Neural Networks
• Theoretical Understanding
• Performance Gap

Innovation

• Graph Neural Networks
• Deeper GNNs
• Message Passing Neural Networks