🤖 AI Summary
This work addresses the limitation of existing high-dimensional complexity measures—such as Gaussian width—which rely on Euclidean geometry and fail to capture the intrinsic Riemannian structure and parametrization invariance of statistical models. We propose Fisher width, a generalization of Gaussian width to the statistical manifold endowed with the Fisher information metric, by rescaling sets via the local metric tensor to reflect statistical distinguishability. This measure provides the first parametrization-invariant geometric complexity tool for statistical manifolds, capturing anisotropic geometric effects overlooked by Euclidean approaches. Leveraging differential geometry, information geometry, and high-dimensional probability, we establish concentration properties, perturbation stability, and spectral comparison bounds for Fisher width, and design a computable estimator. Under a Fisher-Lipschitz assumption, we derive generalization bounds and empirically validate the method’s efficacy across three model classes on MNIST.
📝 Abstract
Gaussian width is a central geometric complexity measure in high-dimensional probability, compressed sensing, convex optimization, and learning theory. It quantifies the average extent of a set along random directions, thereby capturing the effective dimension of constraint sets, hypothesis classes, and descent cones. However, this notion is intrinsically Euclidean. Statistical models instead carry a natural Riemannian geometry induced by the Fisher information metric, where directions are scaled according to statistical distinguishability rather than ambient Euclidean length.
We introduce Fisher width, a Fisher-geometric analogue of Gaussian width for statistical manifolds. At a parameter point $θ$, Fisher width replaces the Euclidean identity by the local metric tensor $G(θ)^{1/2}$, measuring the Gaussian width of the Fisher-rescaled set. This makes the resulting quantity sensitive to local statistical curvature and invariant under smooth reparameterizations.
We develop the basic theory of Fisher width, showing that it retains key structural features of Gaussian width, including concentration, metric perturbation stability, and spectral comparison bounds with the Euclidean baseline, while also capturing anisotropic geometric effects invisible to Euclidean measures. As an application, we prove a generalization bound for Fisher-Lipschitz hypothesis classes and propose computable estimators, which we evaluate empirically on MNIST across three model classes.
Fisher width is to statistical manifolds what Gaussian width is to Euclidean convex bodies. This work lays the foundation for studying complexity and learning on curved statistical manifolds.