Dynamics to decision: A mathematical theory of Lyapunov spectra and decision boundaries in deep classifiers

2026-09-30

AI Summary

This study investigates how the geometric structure of decision boundaries in deep classifiers evolves with layer depth and the dynamical mechanisms governing this process. By modeling feedforward networks as non-autonomous discrete dynamical systems, we employ finite-time maximum Lyapunov exponents (FTMLE) to analyze data trajectories, revealing the dynamical characteristics of decision boundaries across probability, logit, and hidden layers. This work establishes the first mathematical connection between the Lyapunov spectrum and decision boundaries, proving that probability-level FTMLE encodes normal geometric information of the boundary. Building on this insight, we propose a geometry-aware fine-tuning strategy that reconstructs sensitivity distributions within hidden layers, thereby providing a theoretical foundation for layer-aware regularization.

Abstract

A deep classifier is defined not only by the decision it produces, but also by the sequence of transformations through which that decision is formed. Treating this evolution as a dynamical system across layers provides a natural framework for asking how decision geometry emerges through depth and how far back we can trace a boundary's dynamical signature. We model a feed-forward classifier as a finite, nonautonomous discrete dynamical system, with layers playing the role of discrete time steps. We study the Finite-Time Maximum Lyapunov Exponent (FTMLE) of the data samples' dynamical trajectory through depths of the classifier. The FTMLE measures the rate of convergence/divergence of nearby trajectories. We move the observation endpoint backward from probabilities to logits and then to hidden representations. For Gaussian classes, we prove that probability-level FTMLE carries a clear geometric signature of the decision boundary, with its dominant direction aligned with the boundary normal. Moving one step backward to the logits, we prove this relationship is no longer universal but depends critically on how the classifier is trained, particularly on the choice of loss function. Moving further backward to the hidden representation, the connection becomes more conditional: boundary-related FTMLE can persist, but only under identifiable structural conditions. We propose geometry-aware fine-tuning for restructuring the classifier's hidden FTMLE, and propose conditions for guaranteed concentration of high hidden FTMLE near the decision boundary. Through our numerical results, we show the generality and validity of our theoretical results. Understanding the evolution of data samples as traveling through the layers of classifier provides a principled foundation for identifying where boundary-relevant sensitivity emerges and for developing layer-aware regularization strategies.

Problem

• Deep classifiers
• Decision boundaries
• Lyapunov spectra
• Dynamical systems
• Finite-Time Maximum Lyapunov Exponent

Innovation

• Lyapunov exponents
• decision boundaries
• dynamical systems
• deep classifiers
• geometry-aware fine-tuning