Riemannian Gradient Descent for Gaussian Mixture Models with unknown diagonal covariances

2026-09-24

AI Summary

This study addresses the challenging problem of estimating Gaussian Mixture Models (GMMs) with an unknown number of components and diagonal covariance matrices. Within the Fisher-Rao geometric framework, the estimation task is reformulated as a Beurling LASSO (BLASSO) model, and a joint optimization algorithm integrating Conic Particle Gradient Descent (CPGD) with Riemannian gradient descent is proposed. Theoretically, the intrinsic connection between exponential local convergence and non-degeneracy conditions is established, providing rigorous convergence guarantees. Experimentally, compared to the conventional Expectation-Maximization (EM) algorithm, the proposed method demonstrates significantly greater robustness to over-specified component numbers and substantially improves parameter recovery accuracy.

Abstract

This paper investigates the numerical resolution of the Beurling-LASSO (BLASSO), a convex optimization framework that promotes sparsity in the space of measures. We consider its application to the estimation of Gaussian mixture models (GMMs) with an unknown number of components and unknown diagonal covariance matrices. Our approach combines the Conic Particle Gradient Descent (CPGD) principle with Riemannian gradient descent, to account for the underlying Fisher-Rao geometry of Gaussian distributions. Our contributions are twofold. First, we provide theoretical guarantees for the convergence of our algorithm. In particular, we establish exponential local convergence under a non-degeneracy condition on the solution and relate this assumption to a separation condition on the underlying statistical target. Second, we address practical implementation aspects of CPGD and present numerical experiments illustrating its performance. On the test cases considered, these experiments suggest that CPGD is more robust to overspecification of the number of components than the EM algorithm. We also investigate the impact of component separation on recovery accuracy.

Problem

• Gaussian Mixture Models
• Beurling-LASSO
• sparse measure estimation
• unknown covariance
• convex optimization

Innovation

• Riemannian Gradient Descent
• Beurling-LASSO
• Conic Particle Gradient Descent
• Gaussian Mixture Models
• Fisher-Rao Geometry