Debiased Bayesian Inference for High-dimensional Regression Models

2025-12-09

AI Summary

In high-dimensional regression, Bayesian posteriors induced by sparsity-promoting priors (e.g., the horseshoe prior) enjoy desirable sparsity properties but yield credible sets that lack frequentist asymptotic validity. To address this, we propose a global posterior debiasing framework and establish, for the first time, a Bernstein–von Mises theorem for the debiased posterior—thereby unifying Bayesian inference with frequentist guarantees. Our method integrates sparsity-inducing priors, an analytical bias-correction mechanism, and rigorous asymptotic normality analysis. We theoretically prove that the debiased posterior is asymptotically normal and yields credible sets with nominal coverage probability. Monte Carlo simulations and empirical applications in economics confirm that its 95% credible intervals achieve target coverage and that parameter estimates outperform standard benchmarks. This work substantially bridges the gap between Bayesian inferential practice and frequentist statistical validity.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Probabilistic Inference
• Game Theory and Economic Paradigms: Mechanism Design

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

There has been significant progress in Bayesian inference based on sparsity-inducing (e.g., spike-and-slab and horseshoe-type) priors for high-dimensional regression models. The resulting posteriors, however, in general do not possess desirable frequentist properties, and the credible sets thus cannot serve as valid confidence sets even asymptotically. We introduce a novel debiasing approach that corrects the bias for the entire Bayesian posterior distribution. We establish a new Bernstein-von Mises theorem that guarantees the frequentist validity of the debiased posterior. We demonstrate the practical performance of our proposal through Monte Carlo simulations and two empirical applications in economics.

Problem

• Debiasing Bayesian posteriors for high-dimensional regression models
• Ensuring frequentist validity of debiased posterior credible sets
• Correcting bias in Bayesian inference with sparsity-inducing priors

Innovation

• Debiasing approach corrects Bayesian posterior bias
• Bernstein-von Mises theorem ensures frequentist validity
• Method applied to high-dimensional regression with sparsity priors