Model Selection for SLOPE Models: A Bayesian Perspective

2026-06-07

AI Summary

This work addresses the lack of effective methods in existing SLOPE models to simultaneously achieve high predictive performance and rigorous false discovery rate (FDR) control under general design matrices. The authors propose Bayesian Group SLOPE (BGSLOPE) and Bayesian Sparse Group SLOPE (BSGS), which, for the first time, embed the SLOPE penalty within a continuous spike-and-slab Bayesian framework. To restore FDR control in non-orthogonal designs, they introduce a two-step orthogonalization (TSO) strategy. This approach unifies strong statistical power with strict FDR guarantees and enables robust model selection through Bayesian inference and cross-validation. Empirical results on both synthetic and real-world datasets demonstrate that the proposed methods consistently control FDR, substantially improve detection power, and outperform existing approaches in predictive accuracy.

Abstract

Sorted $\ell_1$ Penalized Estimation (SLOPE) models, that perform either variable or group selection, control the false discovery rate (FDR) under orthogonal settings with known noise, but such settings are rare in practice. Under general conditions, cross-validation is the default model selection approach for SLOPE, yet it targets predictive performance rather than FDR control. We address this gap for the SLOPE family of models by proposing new Bayesian approaches, Bayesian Group SLOPE (BGSLOPE) and Bayesian Sparse-group SLOPE (BSGS). BGSLOPE and BSGS embed group-based SLOPE models into a spike-and-slab framework, with BSGS providing a continuous spike-and-slab framework for sparse-group models. We further introduce Two-step Orthogonal (TSO), which transforms a general setting into an orthogonal one to recover SLOPE's FDR control properties. Through extensive synthetic and real data studies comparing all major model selection strategies for SLOPE models, the proposed Bayesian models consistently control FDR, achieve higher power, and outperform competing methods in prediction.

Problem

• SLOPE
• False Discovery Rate
• Model Selection
• Bayesian Methods
• Orthogonal Design

Innovation

• Bayesian SLOPE
• FDR control
• spike-and-slab
• group selection
• Two-step Orthogonal