Density estimation via mixture discrepancy and moments

2025-04-02

AI Summary

Star discrepancy—the gold standard for assessing uniformity in high-dimensional density estimation—suffers from exponential computational complexity and lacks reflection and rotational invariance, hindering its practical use. Method: We propose an adaptive binary sequence partitioning scheme that constructs piecewise-constant density approximations by jointly optimizing mixed discrepancy (DSP-mix) and low-order moment matching (MSP). Contribution/Results: This is the first work to integrate mixed discrepancy with moment matching within a sequence partitioning framework, ensuring both computational tractability (polynomial-time complexity) and strict geometric invariance—addressing fundamental theoretical and algorithmic limitations of star discrepancy. Empirical evaluation on 2–6D Gaussian and Beta mixture distributions shows a ~10× speedup over state-of-the-art methods while maintaining density estimation accuracy comparable to the original DSP approach.

Technology Category

Search and Optimization: Mixed Discrete/Continuous SearchConstraint Satisfaction and Optimization: Mixed Discrete/Continuous OptimizationPlanning, Routing, and Scheduling: Mixed Discrete/Continuous Planning

Application Category

Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphsUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and rankingSecurity and Privacy: Large-scale security measurements

Abstract

With the aim of generalizing histogram statistics to higher dimensional cases, density estimation via discrepancy based sequential partition (DSP) has been proposed [D. Li, K. Yang, W. Wong, Advances in Neural Information Processing Systems (2016) 1099-1107] to learn an adaptive piecewise constant approximation defined on a binary sequential partition of the underlying domain, where the star discrepancy is adopted to measure the uniformity of particle distribution. However, the calculation of the star discrepancy is NP-hard and it does not satisfy the reflection invariance and rotation invariance either. To this end, we use the mixture discrepancy and the comparison of moments as a replacement of the star discrepancy, leading to the density estimation via mixture discrepancy based sequential partition (DSP-mix) and density estimation via moments based sequential partition (MSP), respectively. Both DSP-mix and MSP are computationally tractable and exhibit the reflection and rotation invariance. Numerical experiments in reconstructing the $d$-D mixture of Gaussians and Betas with $d=2, 3, dots, 6$ demonstrate that DSP-mix and MSP both run approximately ten times faster than DSP while maintaining the same accuracy.

Problem

• Generalize histogram statistics to higher dimensions
• Replace star discrepancy with mixture discrepancy and moments
• Improve computational efficiency while maintaining accuracy

Innovation

• Uses mixture discrepancy for density estimation
• Employs moments comparison for partition
• Ensures reflection and rotation invariance