🤖 AI Summary
To address prediction instability and inaccuracy under high-dimensional, strongly correlated covariates, this paper proposes the Random Subset Averaging (RSA) ensemble method: base models are constructed via binomial random sampling, and adaptively fused through two rounds of data-driven weighting—structurally analogous to a two-layer neural network. Theoretical contributions include: (i) the first asymptotic optimality guarantee for two-stage adaptive weighting; (ii) allowance for fully data-dependent weights in the first stage; and (iii) tighter finite-sample risk bounds under orthogonal design. RSA integrates cross-validation-based hyperparameter tuning with dual-stage weighting, ensuring both computational feasibility and statistical rigor. Empirical results demonstrate that RSA significantly outperforms LASSO, random forests, and state-of-the-art ensemble methods across diverse sparse and highly correlated settings. Its practical robustness and superiority are further validated in financial return forecasting.
📝 Abstract
We propose a new ensemble prediction method, Random Subset Averaging (RSA), tailored for settings with many covariates, particularly in the presence of strong correlations. RSA constructs candidate models via binomial random subset strategy and aggregates their predictions through a two-round weighting scheme, resulting in a structure analogous to a two-layer neural network. All tuning parameters are selected via cross-validation, requiring no prior knowledge of covariate relevance. We establish the asymptotic optimality of RSA under general conditions, allowing the first-round weights to be data-dependent, and demonstrate that RSA achieves a lower finite-sample risk bound under orthogonal design. Simulation studies demonstrate that RSA consistently delivers superior and stable predictive performance across a wide range of sample sizes, dimensional settings, sparsity levels and correlation structures, outperforming conventional model selection and ensemble learning methods. An empirical application to financial return forecasting further illustrates its practical utility.