Persistent Gaussian Perturbations Prevent Oversmoothing in Recurrent Graph Neural Networks

2026-07-30

AI Summary

This work addresses the over-smoothing problem in deep graph neural networks, wherein node representations collapse into indistinguishable vectors as network depth increases. To mitigate this issue, the authors propose injecting independent Gaussian noise during iterative message propagation and interpret the model as a stochastic dynamical system. They establish, for the first time from the perspective of stochastic dynamical systems, that persistent Gaussian perturbations guarantee a positive lower bound on the Dirichlet energy at steady state, thereby fundamentally preventing asymptotic over-smoothing. Theoretically, this lower bound is shown to scale proportionally with both the noise variance and the spectral gap of the graph Laplacian. Empirical results corroborate the existence of a stationary distribution and its dependence on noise intensity, validating the theoretical analysis.

Abstract

Oversmoothing is a fundamental limitation of deep graph neural networks (GNNs), where repeated message passing causes node representations to become increasingly similar, eventually collapsing toward a low-dimensional subspace. This phenomenon limits the effective depth of message-passing architectures and motivates the search for mechanisms that preserve representation diversity. In this paper, we study a recurrent graph neural network in which independent Gaussian noise is injected after every propagation step and analyze the resulting architecture as a stochastic dynamical system. Under a standard global contraction assumption on the deterministic update, we prove that the hidden representations form a geometrically ergodic Markov chain admitting a unique invariant probability measure. Our main theoretical result establishes an explicit positive lower bound on the expected stationary Dirichlet energy, proportional to both the noise variance and the spectral gap of the underlying graph. Consequently, the stationary representations cannot collapse onto the constant manifold, providing a rigorous guarantee that asymptotic oversmoothing is prevented in the sense of non-vanishing Dirichlet energy. Our analysis reveals persistent stochastic perturbations as a fundamentally different mechanism for combating oversmoothing, complementing existing deterministic approaches based on residual connections, normalization, and graph rewiring. Finally, numerical experiments on both linear and nonlinear recurrent graph neural networks closely match the theoretical predictions, illustrating the emergence of a stationary distribution and the predicted dependence of the limiting Dirichlet energy on the noise intensity.

Problem

• oversmoothing
• graph neural networks
• representation collapse
• message passing
• Dirichlet energy

Innovation

• Gaussian perturbations
• oversmoothing
• recurrent graph neural networks
• Dirichlet energy
• stochastic dynamical systems