reversible jump mcmc

In high-dimensional Bayesian inference, Gibbs sampling implementations of the Dirichlet–Laplace (DL) prior suffer from systematic bias due to ambiguities in a critical step, causing empirical samples to deviate from the target posterior and undermining its asymptotic shrinkage guarantees. Method: We provide the first rigorous characterization of the conditional posterior structure under the DL prior, identify and resolve implicit sampling ambiguities in the original algorithm, and propose an exact Gibbs sampler provably convergent to the correct posterior. The new scheme preserves computational efficiency while ensuring theoretical correctness for both the normal means model and high-dimensional linear regression. Results: Simulation and real-data experiments demonstrate that the corrected method substantially improves posterior distributional accuracy and variable selection consistency, thereby ensuring faithful finite-sample implementation of the DL prior’s asymptotic theoretical properties.

Conventional Bayesian methods for high-dimensional non-sparse linear models suffer from overreliance on parameter sparsity assumptions and neglect of the spectral structure of covariates. Method: We propose a spectrum-aware Bayesian estimation framework that constructs adaptive priors using eigenvectors of the data covariance matrix—thereby circumventing sparsity requirements—and establish a posterior contraction rate theory proving minimax-optimal estimation rates. To enable scalable uncertainty quantification, we introduce a truncated Gaussian approximation and derive Bernstein–von Mises-type asymptotic normality for the posterior. Contribution/Results: This work provides the first spectrally adaptive Bayesian theoretical foundation for overparameterized, non-sparse settings. It achieves both accurate high-dimensional parameter estimation and reliable statistical inference, bridging a critical gap between spectral learning theory and Bayesian methodology in modern high-dimensional statistics.

This work addresses the challenges of efficiently sampling posterior distributions in Bayesian inversion when confronted with high-dimensional parameter spaces, sparse data, and strong noise, which hinder conventional dimensionality reduction techniques. The authors propose the α-likelihood informed subspace (α-LIS) method, which rigorously extends likelihood-informed subspace (LIS) theory to tempered posteriors with α ∈ [0,1], enabling the construction of a partially informed low-dimensional subspace for effective dimension reduction. By integrating data from multiple tempering levels and incorporating a gradient-free approximation strategy, the approach significantly enhances robustness and sampling efficiency in scenarios where gradients are unavailable or observations are highly noisy. Both theoretical analysis and numerical experiments demonstrate that near-optimal dimension reduction can be achieved with relatively small α values, yielding overall performance superior to traditional methods restricted to α = 1.

This paper addresses dynamic variable selection in high-dimensional time-varying regression with pre-specified group structures. We propose a scalable variational Bayesian framework, the first to integrate variational inference into this setting. Our method jointly incorporates dynamic sparsity-inducing priors—encompassing both group-wise sparsity and time-varying shrinkage—high-dimensional time-series modeling, and efficient approximate posterior computation. It achieves a favorable balance between statistical accuracy and computational scalability, making it suitable for large-scale macroeconomic forecasting tasks, such as inflation modeling. In extensive simulations and empirical analyses using real macroeconomic data, the method delivers substantial improvements in both point and density forecasting accuracy. Moreover, it uncovers economically interpretable, time-varying, and group-structured patterns among inflation drivers—revealing how key determinants evolve and cluster over time.

To address the scalability challenges of Bayesian learning under big data and large models—stemming from high-dimensional posterior approximation—this paper proposes a scalable Bayesian inference framework. Methodologically, it introduces a novel tempered stochastic gradient MCMC perspective, theoretically establishing the asymptotic unbiasedness of deep ensembles. It further provides the first systematic empirical validation of the cold posterior effect in large language models (LLMs), demonstrating improved uncertainty calibration and robustness via Bayesian approximation. Finally, it develops Posteriors, an open-source PyTorch library implementing a unified optimization-and-sampling paradigm, enabling efficient Bayesian inference for models with up to thousands of layers. Experiments across multiple benchmarks and LLM tasks show significant gains in predictive uncertainty calibration and out-of-distribution robustness.

This work addresses the challenge of applying Gaussian processes (GPs) to high-dimensional inputs, where they are prone to the curse of dimensionality, and existing two-stage dimensionality reduction approaches often compromise either predictive accuracy or reliable uncertainty quantification. To overcome this limitation, the authors propose an end-to-end Bayesian joint modeling framework that seamlessly integrates input dimensionality reduction within the GP formulation. By placing a prior on the Stiefel manifold to enforce orthogonality of the projection matrix and employing Riemannian Hamiltonian Monte Carlo for posterior inference along geodesics, the method achieves, for the first time, a unified Bayesian treatment of GP regression and dimensionality reduction. The framework is further extended to deep Gaussian processes to enhance representational capacity. Experimental results demonstrate superior performance over conventional two-stage methods in both prediction accuracy and uncertainty calibration, albeit at increased computational cost.

Posterior inference in Bayesian neural networks is often hindered by symmetries, non-identifiability, and semantic ambiguity in the prior. This work systematically investigates the interplay between over-parameterization and the prior, revealing how their synergy reshapes the geometry of the posterior distribution. Through large-scale posterior sampling and geometric analysis, we identify three key phenomena induced by over-parameterization: posterior balancing, weight redistribution along equiprobable manifolds, and enhanced alignment with the prior. These findings demonstrate that appropriate over-parameterization yields a posterior distribution with clearer structure and stronger prior consistency, substantially improving the interpretability and practical utility of posterior inference in Bayesian neural networks.

This work addresses the computational challenges in existing Bayesian factor models arising from the complex hierarchical structure of ordered shrinkage priors, which hinder efficient posterior inference. To overcome this limitation, we propose a novel Bayesian factorization method based on an $L_{1/2}$ shrinkage prior that preserves the desirable ordered shrinkage property of factor loadings while substantially simplifying the prior architecture. The resulting model admits both exact Gibbs sampling and an efficient variational approximation, achieving a favorable balance between computational efficiency and inferential accuracy. Extensive numerical experiments demonstrate that the proposed approach consistently outperforms state-of-the-art Bayesian factor models in terms of both estimation precision and computational speed.

This work addresses the computational inefficiency in Bayesian semiparametric regression arising from complex design matrix structures. To mitigate this challenge, the authors propose an orthogonalization preprocessing step applied to the design日晚间 matrix prior to iterative inference, combined with a hybrid algorithm integrating Gibbs sampling and coordinate ascent variational inference. This approach reduces computational complexity to quadratic in the number of covariates, substantially accelerating both model fitting and posterior inference. Empirical evaluations across diverse experimental settings demonstrate speedups ranging from 5× to 60× compared to conventional methods, effectively alleviating the computational bottleneck induced by high-dimensional covariates.