🤖 AI Summary
In high-dimensional sparse regimes (large dimension-to-sample ratio), spectral distortion of the sample covariance matrix renders standard whitening ineffective: after whitening, the means of a spherical Gaussian mixture model (GMM) fail to become asymptotically orthogonal, severely degrading latent variable estimation based on higher-order moment tensor decomposition. This paper establishes, for the first time, a rigorous characterization—grounded in random matrix theory—of the asymptotic behavior of inner products between whitened means in high dimensions. Leveraging this analysis, we propose a spectrally corrected whitening matrix that achieves asymptotic orthogonality of the means. This correction restores the decomposability of higher-order moment tensors and significantly improves GMM parameter estimation accuracy. Empirical results demonstrate that the proposed corrected whitening remains robust and effective even in finite-sample settings, thereby overcoming fundamental theoretical and practical limitations of conventional whitening in large-dimensional sparse scenarios.
📝 Abstract
Whitening is a classical technique in unsupervised learning that can facilitate estimation tasks by standardizing data. An important application is the estimation of latent variable models via the decomposition of tensors built from high-order moments. In particular, whitening orthogonalizes the means of a spherical Gaussian mixture model (GMM), thereby making the corresponding moment tensor orthogonally decomposable, hence easier to decompose. However, in the large-dimensional regime (LDR) where data are high-dimensional and scarce, the standard whitening matrix built from the sample covariance becomes ineffective because the latter is spectrally distorted. Consequently, whitened means of a spherical GMM are no longer orthogonal. Using random matrix theory, we derive exact limits for their dot products, which are generally nonzero in the LDR. As our main contribution, we then construct a corrected whitening matrix that restores asymptotic orthogonality, allowing for performance gains in spherical GMM estimation.