Density estimation for Hellinger via minimum-distance estimators: mixtures of Gaussians, log-concave, and more

2026-06-09

AI Summary

This work addresses the problem of efficiently estimating complex probability densities—such as Gaussian mixtures and log-concave densities—from finite samples under the Hellinger distance. The authors extend the minimum distance estimation framework to the Hellinger metric for the first time, establishing a general learning paradigm by constructing associated concept classes and leveraging the reverse data processing inequality. Their main contributions include the development of the first near-linear-time algorithm that achieves nearly optimal sample and computational complexity for one-dimensional Gaussian mixture models and mixtures of log-concave densities.

Abstract

We study the task of density estimation, where we hope to accurately estimate a probability density from $n$ samples. A textbook method for density estimation in total variation distance is the minimum-distance estimator approach, where we conclude both the algorithm and the analysis merely from bounding the VC dimension of a particular concept class (the so-called Yatracos class). While this technique has originally yielded sharp guarantees primarily for total variation distance, in this work we extend the minimum-distance estimator approach for learning within Hellinger distance. Our main observation is that we may produce an analogous recipe for Hellinger (where we only require bounding the VC dimension of a related concept class) by drawing connections to recent results yielding reverse data processing inequalities. This recipe is flexible enough to accommodate fast algorithms originally designed for total variation distance; by modifying the approach of Acharya et al. (2017) we conclude the first near-linear time algorithm for learning classes including univariate mixtures of log-concave densities and mixtures of Gaussians (with arbitrary variances), with near-optimal sample complexity.

Problem

• density estimation
• Hellinger distance
• minimum-distance estimator
• probability density
• sample complexity

Innovation

• minimum-distance estimator
• Hellinger distance
• VC dimension
• near-linear time algorithm
• mixture models