🤖 AI Summary
Conventional Bayesian methods for high-dimensional non-sparse linear models suffer from overreliance on parameter sparsity assumptions and neglect of the spectral structure of covariates.
Method: We propose a spectrum-aware Bayesian estimation framework that constructs adaptive priors using eigenvectors of the data covariance matrix—thereby circumventing sparsity requirements—and establish a posterior contraction rate theory proving minimax-optimal estimation rates. To enable scalable uncertainty quantification, we introduce a truncated Gaussian approximation and derive Bernstein–von Mises-type asymptotic normality for the posterior.
Contribution/Results: This work provides the first spectrally adaptive Bayesian theoretical foundation for overparameterized, non-sparse settings. It achieves both accurate high-dimensional parameter estimation and reliable statistical inference, bridging a critical gap between spectral learning theory and Bayesian methodology in modern high-dimensional statistics.
📝 Abstract
In the field of high-dimensional Bayesian statistics, a plethora of methodologies have been developed, including various prior distributions that result in parameter sparsity. However, such priors exhibit limitations in handling the spectral eigenvector structure of data, rendering estimations less effective for analyzing the over-parameterized models (high-dimensional linear models that do not assume sparsity) developed in recent years. This study introduces a Bayesian approach that employs a prior distribution dependent on the eigenvectors of data covariance matrices without inducing parameter sparsity. We also provide contraction rates of the derived posterior estimation and develop a truncated Gaussian approximation of the posterior distribution. The former demonstrates the efficiency of posterior estimation, whereas the latter facilitates the uncertainty quantification of parameters via a Bernstein--von Mises-type approach. These findings suggest that Bayesian methods capable of handling data spectra and estimating non-sparse high-dimensional parameters are feasible.