Oversmoothing as Representation Degeneracy in Neural Sheaf Diffusion

2026-05-11

AI Summary

This work addresses the over-smoothing problem in neural layer diffusion—where representation degradation leads to loss of discriminative information—by introducing a novel perspective grounded in quiver representation theory. It models cellular layer structures as representations of associated incidence quivers and uncovers the algebraic structure of the harmonic space in the diffusion limit. Over-smoothing is formally characterized for the first time as a degeneration phenomenon in representation geometry. To promote balanced geometric configurations, a moment map regularizer from geometric invariant theory is incorporated. The study further identifies a structural obstruction inherent in equidimensional stalk architectures and demonstrates that breaking symmetry via non-uniform stalk dimensions significantly reduces variance or improves validation performance on heterophilic benchmarks, while also achieving superior adaptive stability under rectangular settings.

Abstract

Neural Sheaf Diffusion (NSD) generalizes diffusion-based Graph Neural Networks by replacing scalar graph Laplacians with sheaf Laplacians whose learned restriction maps define a task-adapted geometry. While the diffusion limit of NSD is known to be the space of global sections, the representation-theoretic structure of this harmonic space remains largely implicit. We develop a quiver-theoretic interpretation of NSD by identifying cellular sheaves on graphs with representations of the associated incidence quiver. Under this correspondence, learned sheaf geometries become points in a finite-dimensional representation space. We show that direct-sum decompositions of the underlying incidence-quiver representation induce decompositions of the harmonic space reached in the diffusion limit. This gives an algebraic interpretation of oversmoothing as representation degeneration: learned sheaves may collapse toward low-complexity summands whose global sections fail to preserve discriminative information. Building on this viewpoint, we connect sheaf diffusion to stability and moment-map principles from Geometric Invariant Theory. We introduce moment-map-inspired regularizers that bias restriction maps toward balanced representation geometries, and identify a structural obstruction in equal-stalk architectures: when $d_v = d_e$, admissibility for learnable stability parameters forces the trivial all-object summand onto a stability wall. Non-uniform stalk dimensions remove this obstruction, making adaptive stability meaningful. Experiments on heterophilic benchmarks are consistent with this mechanism: breaking stalk symmetry can reduce variance or improve validation behavior, and adaptive stability becomes more effective in selected rectangular settings. Overall, our framework reframes oversmoothing as a degeneration phenomenon in the representation geometry underlying learned sheaf diffusion.

Problem

• oversmoothing
• representation degeneracy
• neural sheaf diffusion
• harmonic space
• quiver representation

Innovation

• Neural Sheaf Diffusion
• Representation Degeneracy
• Quiver Representations
• Moment Map Regularization
• Oversmoothing