variational gmm discretization

To address the trade-off between inadequate uncertainty quantification and low computational efficiency in modeling mixed-type (continuous + categorical) data, this paper proposes the first Bayesian mixture modeling framework based on coordinate-ascent variational inference (CAVI). The method employs latent variables to capture data heterogeneity and complex inter-variable dependencies, and is the first to systematically apply variational inference to Bayesian modeling of mixed data. It ensures asymptotic consistency of posterior means while substantially reducing computational overhead compared to MCMC. Theoretical analysis provides convergence guarantees. Experiments on simulated datasets and the NHANES real-world dataset demonstrate that the proposed approach achieves both high accuracy—delivering comprehensive uncertainty quantification—and high efficiency—reducing computation time by one to two orders of magnitude relative to state-of-the-art methods—making it suitable for large-scale mixed-data applications.

Balancing multimodal posterior modeling and computational efficiency remains a key challenge in Bayesian variational inference. To address this, we propose a novel variational inference framework based on isotropic Gaussian mixtures—where all components share identical variances and equal weights. We establish, for the first time, a rigorous optimization theory for this variational family, introducing a synergistic optimization scheme that couples entropy-regularized mirror descent (for mean updates) with Bures metric descent (for variance updates), enabling efficient joint minimization of the KL divergence to the true posterior. Unlike standard Gaussian approximations, our method achieves significantly improved fidelity in capturing multimodal posteriors while retaining low memory footprint and rapid convergence. Numerical experiments across benchmark models demonstrate consistent gains in both accuracy and computational efficiency.

This paper addresses the efficient discrete approximation of Gaussian mixture models (GMMs) under the Wasserstein distance, motivated by dual requirements of quantization accuracy and computational scalability in control and cyber-physical system verification. We propose an enhanced quantization framework that integrates sigma-point sampling with adaptive clustering, enabling robust handling of high-dimensional, large-scale, and degenerate GMMs. A rigorous upper bound on the Wasserstein approximation error is derived, and a modular interface is provided to support customizable approximation schemes. Experiments demonstrate that our method achieves sublinear error convergence while significantly reducing computational overhead—outperforming state-of-the-art approaches in accuracy. The core contribution lies in the first systematic integration of sigma-point mechanisms with Wasserstein quantization theory, thereby unifying theoretical guarantees with practical deployability.

This paper addresses the longstanding limitation in posterior summarization for nonparametric Bayesian mixture models—where inference has predominantly focused on random partition point estimates, neglecting direct inference on the mixing measure itself. We propose a decision-theoretic framework that prioritizes the mixing measure as the primary inferential target. Methodologically, we introduce, for the first time, a model-agnostic variant of the sliced Wasserstein distance, integrated with generalized geodesic projection and optimization on the symmetric positive-definite matrix manifold; leveraging the linear structure of Gaussian mixing measures, our approach delivers coherent point estimates of the mixing measure, density function, and random partition simultaneously. Compared to conventional paradigms, our method preserves statistical validity under complex dependency structures, substantially improves geometric coherence and computational efficiency in posterior summarization, and unifies support for both density estimation and clustering inference.

Gibbs sampling for Bayesian mixture models suffers from slow mixing in the marginal posterior over component assignments and struggles to jointly perform model selection and parameter inference. Method: We propose two novel joint-sampling MCMC algorithms: (1) a collapsed Gibbs sampler incorporating unconventional move sets, and (2) a prior-driven, rejection-free component allocation sampler. Both methods jointly update observation assignments and the number of components, unifying model fitting and dimensionality inference. Contribution/Results: Our approaches eliminate the need for post-hoc model selection and substantially improve Markov chain mixing efficiency. In latent class analysis tasks, they reduce mixing time by several-fold compared to state-of-the-art methods while achieving comparable or superior posterior inference accuracy. The framework provides an efficient, fully automated computational solution for high-dimensional Bayesian nonparametric modeling.

This study addresses the challenge of statistical inference for multimodal geometric distributions in complex biological systems, particularly when density functions are defined only up to a normalizing constant. Building upon variational inference, this work integrates black-box variational inference, multiple importance sampling ELBO, and flow matching techniques to construct a highly expressive inference framework tailored for spatial transcriptomics. Key contributions include proposing CoLN, an unnormalized target density; revealing and refuting the conventional belief regarding performance gains of mixture models in variational inference; and developing a hybrid approach that combines multi-marginal flow matching with variational interpolation. Ultimately, this framework substantially enhances both the efficiency of approximate inference and the analytical capability for multimodal biological data, such as three-dimensional spatial transcriptomics.

This study addresses three key challenges in applying Bayesian finite mixture models to equipment degradation risk clustering—sparse signals, unstable clustering, and computational infeasibility of MCMC—by proposing an efficient and stable heterogeneous degradation risk clustering framework. The core innovations include the first empirical validation of the critical role of 8-state global percentile discretization in enhancing model stability; the construction of a 30-dimensional multi-source feature engineering pipeline integrating statistical, continuous, and semantic features (including PCA-compressed text embeddings); and the design of an interpretable, anti-overfitting model selection criterion based on WAIC, augmented with constraints on minimum cluster proportion and separation. Replacing NUTS with full-rank automatic differentiation variational inference (ADVI), the approach achieves 84× speedup over NUTS while maintaining stable results, shows 15× faster convergence with high estimation consistency under random-effects models, and accurately identifies the optimal interpretable risk clusters, as demonstrated on a dataset of 280 industrial pumps and 104,703 inspection records.

This work addresses the computational inefficiency often encountered in enriched Dirichlet process mixture models within Bayesian nonparametric inference, particularly when employing complex MCMC algorithms or handling large-scale data. The authors propose an improved truncation approximation strategy integrated with variational Bayes, which substantially simplifies model implementation and accelerates inference. The resulting variational solution serves as a high-quality initialization for Gibbs sampling and is further enhanced by combining blocked Gibbs updates with Pólya urn sampling schemes, enabling efficient implementation within the Nimble platform. Experimental results demonstrate that the proposed approach achieves substantial gains in computational efficiency and practical usability while preserving inferential accuracy.

This work addresses the challenge of intractable posterior distributions in Bayesian inference, where traditional Markov chain Monte Carlo (MCMC) methods are computationally expensive and variational inference (VI) often sacrifices accuracy for efficiency. The paper presents the first systematic integration of VI and MCMC, introducing two novel algorithms: one leverages Gaussian VI to optimize the linear transformation matrix in Hamiltonian Monte Carlo (HMC), thereby enhancing sampling efficiency; the other combines variational autoencoders with Metropolis–Hastings sampling to improve exploration of multimodal posteriors. Experimental results demonstrate that the proposed methods significantly outperform standard MCMC on high-dimensional, complex distributions, effectively capturing all posterior modes while maintaining both computational efficiency and estimation accuracy.

本文提出了一种在贝叶斯希尔伯特空间中分析和分解随机密度混合物的方法，采用惩罚最大似然法并结合坐标式最大化算法实现。