CoRe-GNN: Multilevel Message passing on Coarsened graphs

2026-08-03

AI Summary

This work addresses the high memory overhead and limited expressiveness of existing methods in large-scale graph neural network (GNN) training, which often fail to simultaneously capture long-range dependencies and preserve node distinguishability. To overcome these limitations, we propose CoRe-GNN, a novel framework that unifies cross-cluster and intra-cluster message passing in parallel: the former leverages graph coarsening to model long-range structural information, while the latter retains fine-grained local node features. By integrating cluster-based batching, CoRe-GNN enables memory-efficient and scalable training with theoretical guarantees on spectral approximation. Empirical results demonstrate that our method consistently outperforms strong baselines such as Cluster-GCN across homophilic, heterophilic, and long-range graphs, achieving state-of-the-art accuracy—particularly in long-range scenarios—while maintaining superior memory efficiency.

Abstract

Training Graph Neural Networks on large graphs is challenged by the memory cost of storing all node representations across layers. We show that several existing scalable approaches can be written as structured modifications of the GNN propagation matrix, providing a unified perspective that exposes their respective limitations. In particular, graph coarsening replaces it by a low-rank approximation that enables spectral guarantees but assigns uniform representations to clustered nodes, while Cluster-GCN restricts the propagation matrix to intra-cluster connections that allow efficient batching but sever long-range information. These are complementary failures of the \emph{same} decomposition of the graph into groups of nodes. To obtain the best of both worlds, we propose \textbf{CoRe-GNN}, which performs both propagations in parallel at each layer: a coarsened inter-cluster term capturing long-range structure, and a local intra-cluster term preserving per-node discriminability. We prove that CoRe-GNN inherits analogous approximation guarantees to those of graph coarsening, and introduce a natural cluster-based \emph{batching scheme} that scales to graphs with millions of nodes. On node classification benchmarks spanning homophilic, heterophilic, large-scale, and long-range graphs, CoRe-GNN outperforms both graph coarsening and Cluster-GCN baselines. Notably, CoRe-GNN reaches competitive accuracy on \emph{long-range} tasks, while remaining memory-efficient through batching.

Problem

• Graph Neural Networks
• Large-scale Graphs
• Memory Efficiency
• Graph Coarsening
• Long-range Information

Innovation

• CoRe-GNN
• graph coarsening
• message passing
• scalable GNNs
• cluster-based batching