🤖 AI Summary
This work investigates perturbation stability of deep learning systems under representation-parameter coupling dynamics. Methodologically, it introduces the *Learning Stability Profile*—a novel concept—and establishes a unified variational and Lyapunov functional analysis framework capable of characterizing stability for both smooth and nonsmooth systems, including those with ReLU activations and subgradient flows. Theoretically, it rigorously proves that bounded stability is equivalent to energy dissipation, derives an explicit stability index formula, and reveals connections to spectral stability, the CFL condition in ResNets, and temporal stability laws in stochastic optimization. The approach integrates Clarke’s generalized derivatives, variational inequalities, and continuous-time learning flow modeling. These contributions provide a unifying explanation of robustness mechanisms across diverse architectures and optimizers, and lay foundational groundwork for geometric learning dynamics and continuous-limit theories of deep learning.
📝 Abstract
We propose a unified analytic and variational framework for studying stability in deep learning systems viewed as coupled representation-parameter dynamics. The central object is the Learning Stability Profile, which tracks the infinitesimal response of representations, parameters, and update mechanisms to perturbations along the learning trajectory. We prove a Fundamental Analytic Stability Theorem showing that uniform boundedness of these stability signatures is equivalent, up to norm equivalence, to the existence of a Lyapunov-type energy that dissipates along the learning flow. In smooth regimes, the framework yields explicit stability exponents linking spectral norms, activation regularity, step sizes, and learning rates to contractivity of the learning dynamics. Classical spectral stability results for feedforward networks, a discrete CFL-type condition for residual architectures, and parametric and temporal stability laws for stochastic gradient methods arise as direct consequences. The theory extends to non-smooth learning systems, including ReLU networks, proximal and projected updates, and stochastic subgradient flows, by replacing classical derivatives with Clarke generalized derivatives and smooth energies with variational Lyapunov functionals. The resulting framework provides a unified dynamical description of stability across architectures and optimization methods, clarifying how architectural and algorithmic choices jointly govern robustness and sensitivity to perturbations. It also provides a foundation for further extensions to continuous-time limits and geometric formulations of learning dynamics.