🤖 AI Summary
This study addresses the challenges of calibrating Bayesian hierarchical priors and the computational inefficiency of posterior inference in sparse regression. To this end, it proposes an empirical Bayes framework grounded in convex penalties and log-concave priors. Methodologically, the approach leverages a unified proximal structure to optimize Monte Carlo sampling for efficient posterior uncertainty quantification, while introducing empirical Bayes calibration under affine information and geometry-aware posterior preconditioning techniques. Experiments on synthetic data demonstrate that the proposed method substantially improves both signal recovery accuracy and sampling efficiency. Furthermore, evaluations on a diabetes dataset validate the reliability of the estimated posterior uncertainties and highlight their practical utility in facilitating sparse decision-making.
📝 Abstract
Sparse regression requires both estimation and a decision about which effects to retain. Bayesian shrinkage supplies uncertainty for that decision, but richer prior hierarchies can make calibration and posterior computation demanding. We develop a computationally efficient empirical Bayes framework for Gaussian sparse regression based on convex penalties and log-concave priors. The observation scale and global shrinkage parameter are calibrated separately, the mode summarises information from the joint posterior and provides coefficient estimates, and a proximal sampler supplies posterior uncertainty. A posterior-scale magnitude threshold and activation probability then convert these outputs into a sparse decision. The same proximal structure is reused throughout, making optimisation and sampling inexpensive and allowing extensions to other convex penalties with tractable proximal maps. We also develop empirical Bayes calibration under affine information, distinguishing hard homogeneous constraints from soft nonhomogeneous affine information, and introduce geometry-aware posterior preconditioning when strong affine information creates low-rank stiffness. Synthetic experiments show accurate recovery when the sample size exceeds the number of predictors, conservative weak-signal behaviour when predictors outnumber observations, and substantial gains in Monte Carlo efficiency from geometry-aware scaling. On a diabetes dataset, posterior uncertainty agrees closely with established Bayesian analyses while the terminal decision provides a sparser practical summary.