🤖 AI Summary
Gibbs sampling for Bayesian mixture models suffers from slow mixing in the marginal posterior over component assignments and struggles to jointly perform model selection and parameter inference. Method: We propose two novel joint-sampling MCMC algorithms: (1) a collapsed Gibbs sampler incorporating unconventional move sets, and (2) a prior-driven, rejection-free component allocation sampler. Both methods jointly update observation assignments and the number of components, unifying model fitting and dimensionality inference. Contribution/Results: Our approaches eliminate the need for post-hoc model selection and substantially improve Markov chain mixing efficiency. In latent class analysis tasks, they reduce mixing time by several-fold compared to state-of-the-art methods while achieving comparable or superior posterior inference accuracy. The framework provides an efficient, fully automated computational solution for high-dimensional Bayesian nonparametric modeling.
📝 Abstract
We describe two Monte Carlo algorithms for sampling from the integrated posterior distributions of a range of Bayesian mixture models. Both algorithms allow us to directly sample not only the assignment of observations to components but also the number of components, thereby fitting the model and performing model selection over the number of components in a single computation. The first algorithm is a traditional collapsed Gibbs sampler, albeit with an unusual move-set; the second builds on the first, adding rejection-free sampling from the prior over component assignments, to create an algorithm that has excellent mixing time in typical applications and outperforms current state-of-the-art methods, in some cases by a wide margin. We demonstrate our methods with a selection of applications to latent class analysis.