🤖 AI Summary
This study addresses the asymptotic behavior of Bayesian posteriors in high-dimensional generalized linear models when the number of features grows proportionally with the sample size. Despite concerns about posterior contraction failure and unclear finite-dimensional marginal behavior in this regime, the authors establish—via leave-one-out analysis, high-dimensional asymptotic theory, and explicit posterior derivations—that the marginal posterior converges to a prior-induced Gaussian tilt distribution, whose mean depends on the true signal coordinates. This result demonstrates the persistent influence of the prior in high-dimensional Bayesian inference and further shows that the posterior mean strictly dominates the maximum likelihood estimator in terms of mean squared error, uniformly across all sparsity levels.
📝 Abstract
We investigate Bayes posterior distributions in high-dimensional generalized linear models (GLMs) under the proportional asymptotics regime, where the number of features and samples diverge at a comparable rate. Specifically, we characterize the limiting behavior of finite-dimensional marginals of the posterior. We establish that the posterior does not contract in this setting. Yet, the finite-dimensional posterior marginals converge to Gaussian tilts of the prior, where the mean of the Gaussian depends on the true signal coordinates of interest. Notably, the effect of the prior survives even in the limit of large samples and dimensions. We further characterize the behavior of the posterior mean and demonstrate that the posterior mean can strictly outperform the maximum likelihood estimate in mean-squared error in natural examples. Importantly, our results hold regardless of the sparsity level of the underlying signal. On the technical front, we introduce leave-one-out strategies for studying these marginals that may be of independent interest for analyzing low-dimensional functionals of high-dimensional signals in other Bayesian inference problems.