connect shrinkage to priors

Conventional shrinkage estimation for fixed-effect coefficients in high-dimensional generalized linear models (GLMs) often relies on strong structural assumptions—particularly sparsity—which may not hold in practice. Method: We propose a universal regularization framework that dispenses with sparsity assumptions. Our approach introduces, for the first time, a permutation-invariant ideal prior and couples it with a Pólya tree nonparametric prior to adaptively model the empirical cumulative distribution function (CDF) of the regression coefficients within a hierarchical Bayesian framework, yielding posterior mean estimates. Contribution/Results: The method is theoretically general, requiring no assumptions about coefficient structure. Extensive simulations and real-data experiments demonstrate that it consistently outperforms Lₚ regularization, penalized likelihood methods, and state-of-the-art Bayesian high-dimensional regression approaches in both estimation accuracy and predictive performance. This work establishes a new paradigm for high-dimensional GLMs—one that is simultaneously flexible, assumption-robust, and principled.

This work addresses Bayesian shrinkage estimation for the mean of a multivariate normal distribution under multiple prior target information. It proposes a general construction that, for the first time, explicitly yields a proper Bayes minimax multiple-shrinkage estimator. The method leverages a mixture structure based on square-root-harmonic marginal densities to adaptively shrink toward the more favorable among several prespecified targets, thereby overcoming limitations of existing approaches that rely on either a single target or improper priors. The resulting estimator possesses minimaxity, uniformly dominating the maximum likelihood estimator in terms of risk and achieving substantial risk reduction over a broad region of the parameter space, thus enhancing both robustness and efficiency.

This paper addresses computational and modeling bottlenecks in Bayesian regularization for multivariate statistical models. Methodologically, it introduces a unified Bayesian framework integrating parameter quantization (rounding) and approximate inference: shrinkage estimators—such as those shrinking toward a common mean—are rigorously interpreted as posterior means under specific priors, and an efficient approximate Bayesian algorithm is developed to balance accuracy and scalability. Theoretically, it establishes a rigorous connection between Bayesian inference and numerical approximation, providing the first unified Bayesian interpretation of classical shrinkage strategies. Applicationally, the framework is extended to regularized Linear Discriminant Analysis (LDA), yielding substantial improvements in classification stability and parameter estimation accuracy on both synthetic and real-world datasets. The approach thus bridges theoretical rigor with practical efficiency in high-dimensional multivariate inference.

In high-dimensional Bayesian inference, Gibbs sampling implementations of the Dirichlet–Laplace (DL) prior suffer from systematic bias due to ambiguities in a critical step, causing empirical samples to deviate from the target posterior and undermining its asymptotic shrinkage guarantees. Method: We provide the first rigorous characterization of the conditional posterior structure under the DL prior, identify and resolve implicit sampling ambiguities in the original algorithm, and propose an exact Gibbs sampler provably convergent to the correct posterior. The new scheme preserves computational efficiency while ensuring theoretical correctness for both the normal means model and high-dimensional linear regression. Results: Simulation and real-data experiments demonstrate that the corrected method substantially improves posterior distributional accuracy and variable selection consistency, thereby ensuring faithful finite-sample implementation of the DL prior’s asymptotic theoretical properties.

In Bayesian modeling, expert priors are often specified directly on observable or derived quantities (e.g., survival rates, R²), yet translating such domain knowledge into informative priors for latent model parameters remains a fundamental challenge. Method: We propose a hyperparameter optimization framework grounded in prior predictive distribution matching. It parameterizes a prior family and employs multi-stage Bayesian global optimization to minimize the discrepancy between the induced prior predictive distribution and the target expert-specified distribution—supporting mixed-type and nonstandard targets, as well as censored data, nonlinear structures, and complex derived statistics. Contribution/Results: Across three case studies—cure-rate survival models, R²-driven modeling, and nonlinear regression—the resulting informative priors substantially improve posterior stability and interpretability. Our approach provides the first systematic, computationally tractable, and empirically verifiable methodology for transforming marginal expert beliefs about observables into joint priors over latent parameters.

This work addresses the computational challenges in existing Bayesian factor models arising from the complex hierarchical structure of ordered shrinkage priors, which hinder efficient posterior inference. To overcome this limitation, we propose a novel Bayesian factorization method based on an $L_{1/2}$ shrinkage prior that preserves the desirable ordered shrinkage property of factor loadings while substantially simplifying the prior architecture. The resulting model admits both exact Gibbs sampling and an efficient variational approximation, achieving a favorable balance between computational efficiency and inferential accuracy. Extensive numerical experiments demonstrate that the proposed approach consistently outperforms state-of-the-art Bayesian factor models in terms of both estimation precision and computational speed.

This work addresses the lack of a unified framework for shrinkage, thresholding, and regularization methods in normal mean estimation by proposing a general class of estimators that encompasses both James–Stein-type and Lasso-type rules. Within this class, the authors derive the NOMAD estimator by minimizing a feasible, data-driven approximation to the risk. The approach is extended to settings with correlated observations and linear regression. A key innovation is the establishment of a unified risk minimization framework featuring a data-adaptive penalty, which—remarkably—achieves approximate risk consistency in correlated normal mean models for the first time. Theoretical analysis confirms that the estimator is consistent under both independent and correlated designs, admits an equivalent penalized regression formulation, and either subsumes or improves upon several classical methods in theory.

This work addresses the high computational cost of Bayesian inference with projection priors, which typically require nested iterative optimization. The authors propose a continuously relaxed projection prior that eliminates the need for inner-loop optimization during posterior updates by introducing a duality gap and placing a probabilistic prior on it to drive its contraction toward zero. This approach is the first to incorporate a continuous shrinkage mechanism into projection priors, yielding a differentiable, inner-loop-free approximate prior. The method also establishes a theoretical connection to global–local shrinkage priors. Empirical results demonstrate competitive posterior shrinkage performance, and the approach is successfully applied to marketing data analysis of multivariate shopping decisions, effectively identifying key predictive factors.

This work addresses the lack of explicit confidence modeling for various sources of uncertainty in Bayesian inference by proposing a general extension framework that, for the first time, explicitly incorporates confidence in key uncertainty components—such as the prior and likelihood—into Bayesian modeling. The framework not only introduces a novel regularization mechanism but also provides a unified approach to inducing model sparsity. Without compromising theoretical rigor, the method achieves controllable sparsity across diverse models, including linear regression, logistic regression, and Bayesian neural networks, thereby significantly enhancing both interpretability and generalization performance.

This work investigates the unintended consequences of element-wise truncation in enforcing positive definiteness for symmetric positive definite matrix priors. While such truncation preserves positive definiteness, it systematically favors sparser structures under sparsity-inducing priors, thereby compromising prior interpretability, shrinkage properties, and posterior reliability. Through probabilistic truncation analysis, high-dimensional asymptotic theory, and sparse Bayesian inference, the study characterizes how truncation perturbs both dense and sparse priors and their marginal distributions. The key contribution lies in uncovering the structural bias induced by truncation in high dimensions and proposing a dimension-adaptive calibration strategy for prior parameters. This approach preserves positive definiteness while maintaining semantic consistency of the prior, substantially enhancing the stability and reliability of Bayesian modeling for sparse positive definite matrices.