🤖 AI Summary
This work proposes a smooth geometric framework based on diffusion Markov operators to stably characterize the geometric structure of intermediate representations in feedforward neural networks, enabling a unified analysis of their separability, contraction properties, and generalization capacity. By leveraging a Gaussian kernel-induced diffusion process together with Bakry–Émery Γ-calculus, the framework constructs continuous and perturbation-smooth observables—such as transport distances, spectral characteristics, class boundaries, and local scales—thereby overcoming the discontinuities inherent in traditional neighborhood graph approaches. Under Gaussian class-conditional assumptions, the authors derive a closed-form solution for Gaussian bridges and validate the framework on MNIST, demonstrating its effectiveness in tracking training dynamics, the impact of network width, and robustness to input perturbations.
📝 Abstract
Neural networks transform data through learned representations whose geometry affects separation, contraction, and generalization. Recent work studies this geometry using discrete curvature on neighborhood graphs, suggesting Ricci-flow-like behavior across layers. We develop a smooth operator-theoretic alternative for feedforward representation snapshots. Each feature cloud induces a Gaussian-kernel diffusion Markov operator, and transport, spectral, label-boundary, and local-scale observables are derived from this single object via Bakry-Emery $Γ$-calculus. In a balanced Gaussian class-conditional snapshot model with shared covariance, the population operator has closed-form class affinities, leakage, and coarse spectra, all controlled by pairwise regularized Mahalanobis separations $c_\varepsilon^{(a,b)}$. We also prove that the resulting operator observables vary smoothly under feature perturbations, while hard neighborhood-graph diagnostics can change discontinuously. Synthetic experiments validate the closed-form Gaussian bridge, while learned MNIST experiments show that the same operator observables track training, width, and perturbation stability. Together, these results give a stable operator-geometric framework for analyzing feedforward representation geometry.