🤖 AI Summary
This paper investigates the structural properties of the family ℱₘ,α of empirical distributions induced by all *m*-dimensional linear projections of high-dimensional Gaussian data, where *n*, *d* → ∞ and *n*/*d* → α. Using tools from random matrix theory, optimal transport, and information theory, we (i) precisely characterize the Wasserstein radius of ℱₘ,α—fully solving the *m* = 1 case—and derive tight upper and lower bounds on its KL divergence and Rényi information dimension; (ii) extend classical unsupervised projection analysis to supervised learning settings, establishing sharp Wasserstein radius bounds; and (iii) derive a tight upper bound on the interpolation threshold for two-layer neural networks with *m* hidden neurons. Collectively, these results provide a unified theoretical framework and fundamental benchmarks for high-dimensional projection statistics, representation learning, and the analysis of overparameterized models.
📝 Abstract
Given a cloud of $n$ data points in $mathbb{R}^d$, consider all projections onto $m$-dimensional subspaces of $mathbb{R}^d$ and, for each such projection, the empirical distribution of the projected points. What does this collection of probability distributions look like when $n,d$ grow large? We consider this question under the null model in which the points are i.i.d. standard Gaussian vectors, focusing on the asymptotic regime in which $n,d oinfty$, with $n/d oalphain (0,infty)$, while $m$ is fixed. Denoting by $mathscr{F}_{m, alpha}$ the set of probability distributions in $mathbb{R}^m$ that arise as low-dimensional projections in this limit, we establish new inner and outer bounds on $mathscr{F}_{m, alpha}$. In particular, we characterize the Wasserstein radius of $mathscr{F}_{m,alpha}$ up to constant multiplicative factors, and determine it exactly for $m=1$. We also prove sharp bounds in terms of Kullback-Leibler divergence and R'{e}nyi information dimension. The previous question has application to unsupervised learning methods, such as projection pursuit and independent component analysis. We introduce a version of the same problem that is relevant for supervised learning, and prove a sharp Wasserstein radius bound. As an application, we establish an upper bound on the interpolation threshold of two-layers neural networks with $m$ hidden neurons.
