probabilistic modeling

To address the scalability challenges of Bayesian learning under big data and large models—stemming from high-dimensional posterior approximation—this paper proposes a scalable Bayesian inference framework. Methodologically, it introduces a novel tempered stochastic gradient MCMC perspective, theoretically establishing the asymptotic unbiasedness of deep ensembles. It further provides the first systematic empirical validation of the cold posterior effect in large language models (LLMs), demonstrating improved uncertainty calibration and robustness via Bayesian approximation. Finally, it develops Posteriors, an open-source PyTorch library implementing a unified optimization-and-sampling paradigm, enabling efficient Bayesian inference for models with up to thousands of layers. Experiments across multiple benchmarks and LLM tasks show significant gains in predictive uncertainty calibration and out-of-distribution robustness.

This work addresses the core challenge of accurately inferring high-dimensional, non-convex posterior distributions in Bayesian neural networks and deep generative models. To this end, it systematically establishes the first comprehensive methodology framework for Bayesian approximate inference tailored to deep learning. The framework unifies variational inference, Markov chain Monte Carlo (including stochastic gradient samplers), Laplace approximation, and probabilistic programming into a coherent classification taxonomy and practical paradigm—thereby bridging Bayesian computation with modern deep architectures. The proposed methods substantially improve both posterior approximation accuracy and computational efficiency. Empirically validated across diverse deep Bayesian models, they deliver a theoretically principled yet engineering-practical inference toolkit for trustworthy AI systems.

Statistical analysis of black-box generative models—whose weights, pretraining data, and model covariates are inaccessible—remains challenging due to the absence of internal model information. Method: This paper introduces a data-centric kernel embedding framework that maps each generative model into a reproducing kernel Hilbert space (RKHS) induced by its output sample distribution, yielding model-level comparable representations. The method integrates functional-space projection, maximum mean discrepancy (MMD)-based distributional distance estimation, and nonparametric hypothesis testing to enable interpretable, cross-model statistical inference without requiring internal model access. Contribution/Results: It is the first approach to achieve purely input–output behavior-driven kernel-space embedding of generative models, circumventing black-box constraints. Evaluated on model clustering, anomaly detection, and performance attribution, it significantly outperforms baselines while exhibiting strong generalizability and plug-and-play applicability. This work establishes a novel, covariate-free paradigm for evaluating generative models under strict black-box conditions.

This work addresses the computational intractability of posterior inference in Bayesian neural networks, which hinders scalability. Traditional sequential Monte Carlo (SMC) methods rely on full-batch data, incurring prohibitive computational costs. To overcome this limitation, the authors propose a data annealing strategy that incrementally incorporates mini-batches within the SMC framework, enabling progressive updates to the likelihood and gradient estimates. This approach represents the first effective integration of mini-batch processing with SMC sampling. By doing so, it achieves substantial gains in computational efficiency while preserving sampling accuracy. Empirical evaluations on standard image classification benchmarks demonstrate up to a six-fold speedup compared to conventional SMC, with negligible degradation in model accuracy.

This work addresses the tractability of exact inference and learning in exponential-family latent variable models (LVMs), seeking to characterize the precise boundary of models admitting closed-form analytical solutions without approximation. Method: We derive necessary and sufficient conditions for prior–posterior conjugacy in exponential-family LVMs, providing the first systematic characterization of exact solvability. We further propose a composable graphical model construction framework that preserves structural flexibility while guaranteeing analytic tractability throughout. A general-purpose exact Bayesian inference and parameter learning algorithm is developed, accompanied by an open-source implementation supporting empirical validation across diverse models. Contribution/Results: Our results substantially broaden the class of LVMs amenable to exact inference—bypassing variational approximations or Monte Carlo sampling—and establish a rigorous theoretical foundation and practical toolkit for interpretable, high-precision latent-variable modeling.

This work addresses the lack of explicit confidence modeling for various sources of uncertainty in Bayesian inference by proposing a general extension framework that, for the first time, explicitly incorporates confidence in key uncertainty components—such as the prior and likelihood—into Bayesian modeling. The framework not only introduces a novel regularization mechanism but also provides a unified approach to inducing model sparsity. Without compromising theoretical rigor, the method achieves controllable sparsity across diverse models, including linear regression, logistic regression, and Bayesian neural networks, thereby significantly enhancing both interpretability and generalization performance.

This work proposes a probabilistic inference framework that integrates inductive biases to address the challenges of uncertainty quantification in deep sequential models. While traditional Bayesian approaches struggle with prior specification and inference accuracy in large-scale networks, the proposed method establishes a theoretical connection between Transformer attention mechanisms and sparse Gaussian processes, enabling scalable approximate Bayesian inference. It introduces cross-domain inducing points derived from HiPPO operators to support long-range historical modeling in online learning settings. Furthermore, self-supervised signals are leveraged to enrich the probabilistic structure of latent variables in sequence generation. The resulting approach significantly enhances the uncertainty quantification capability, probabilistic expressiveness, and scalability of deep sequential models, all while maintaining competitive predictive performance.

This work addresses the limitations of traditional variational inference, which struggles to calibrate posterior distributions due to the representational constraints of the evidence lower bound (ELBO). The authors propose a novel single-parameter variational objective that introduces, for the first time, an adjustable score-based posterior, thereby establishing a flexible variational framework capable of jointly learning hierarchical structures and Bayesian posteriors. By incorporating analytically tractable gradient computation, the method significantly improves posterior calibration in mixture models and achieves higher ELBO values in variational autoencoders (VAEs). This enhancement promotes better alignment between the decoder and the prior distribution, ultimately strengthening the model's probabilistic representation capabilities.

This work addresses the inherent uncertainty in 3D scene reconstruction from limited observations—such as a single view, sparse pixels, or noisy images—by proposing a probabilistic framework that integrates Neural Radiance Fields (NeRF) with score-based diffusion models. The method represents the 3D scene as a stochastic latent variable, employs NeRF to model the likelihood of observations, and leverages a diffusion model to learn the prior distribution over the latent space. Crucially, it introduces, for the first time, a score-based diffusion mechanism to sample from the posterior distribution of the latent variables, enabling a unified treatment of uncertainty across diverse observation conditions. A two-stage training strategy jointly optimizes the reconstruction and prior networks, achieving high-fidelity 3D reconstructions under various settings—including single-view, multi-view, noisy images, sparse pixels, and depth inputs—while faithfully capturing task-specific uncertainties.