mHC-GNN: Manifold-Constrained Hyper-Connections for Graph Neural Networks

2026-01-05

arXiv.org

AI Summary

This work addresses the limitations of deep graph neural networks (GNNs), which suffer from over-smoothing and are constrained in expressive power by the 1-Weisfeiler-Lehman (1-WL) test. To overcome these issues, the authors propose a manifold-constrained hyperconnection mechanism that constructs multiple parallel representation streams and employs Sinkhorn–Knopp normalization to constrain the stream mixing matrix to the Birkhoff polytope. This approach effectively mitigates over-smoothing and surpasses the 1-WL expressiveness barrier. Notably, it is the first to integrate manifold-constrained hyperconnections into GNNs, achieving consistent performance gains across ten benchmark datasets and four mainstream GNN architectures. Remarkably, the model maintains over 74% accuracy even at a depth of 128 layers, representing an improvement of more than 50 percentage points over standard GNNs.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Graph Neural Networks (GNNs) suffer from over-smoothing in deep architectures and expressiveness bounded by the 1-Weisfeiler-Leman (1-WL) test. We adapt Manifold-Constrained Hyper-Connections (\mhc)~\citep{xie2025mhc}, recently proposed for Transformers, to graph neural networks. Our method, mHC-GNN, expands node representations across $n$ parallel streams and constrains stream-mixing matrices to the Birkhoff polytope via Sinkhorn-Knopp normalization. We prove that mHC-GNN exhibits exponentially slower over-smoothing (rate $(1-\gamma)^{L/n}$ vs.\ $(1-\gamma)^L$) and can distinguish graphs beyond 1-WL. Experiments on 10 datasets with 4 GNN architectures show consistent improvements. Depth experiments from 2 to 128 layers reveal that standard GNNs collapse to near-random performance beyond 16 layers, while mHC-GNN maintains over 74\% accuracy even at 128 layers, with improvements exceeding 50 percentage points at extreme depths. Ablations confirm that the manifold constraint is essential: removing it causes up to 82\% performance degradation. Code is available at \href{https://github.com/smlab-niser/mhc-gnn}{https://github.com/smlab-niser/mhc-gnn}

Problem

• over-smoothing
• expressiveness
• Graph Neural Networks
• 1-Weisfeiler-Leman test
• deep architectures

Innovation

• Manifold-Constrained Hyper-Connections
• Over-smoothing Mitigation
• Graph Neural Networks
• Birkhoff Polytope
• 1-WL Expressiveness