🤖 AI Summary
This paper addresses the unified modeling of parameter pattern recovery in multi-class sparse regularized estimation (e.g., LASSO, SLOPE). Existing frameworks lack a general characterization of pattern structure and complexity across diverse polyhedral norms. Method: We propose a generalized pattern definition and complexity measure based on polyhedral norms and subdifferentials; extend LASSO’s irrepresentability condition to a geometric, noiseless recovery condition applicable to arbitrary polyhedral norms; and identify “accessibility”—a strictly weaker condition than irrepresentability—as sufficient for deterministic pattern recovery via thresholding estimators. Contributions: (i) A universal necessary and sufficient condition system for pattern recovery; (ii) Proof that the noiseless recovery condition serves as a unifying foundation for pattern identification across all polyhedral-regularized estimators; (iii) Demonstration that thresholding strategies substantially relax recovery requirements and enhance robustness, thereby unifying theory and enabling milder conditions.
📝 Abstract
We consider the framework of penalized estimation where the penalty term is given by a real-valued polyhedral gauge, which encompasses methods such as LASSO, generalized LASSO, SLOPE, OSCAR, PACS and others. Each of these estimators can uncover a different structure or ``pattern'' of the unknown parameter vector. We define a novel and general notion of patterns based on subdifferentials and formalize an approach to measure pattern complexity. For pattern recovery, we provide a minimal condition for a particular pattern to be detected by the procedure with positive probability, the so-called accessibility condition. Using our approach, we also introduce the stronger noiseless recovery condition. For the LASSO, it is well known that the irrepresentability condition is necessary for pattern recovery with probability larger than $1/2$ and we show that the noiseless recovery plays exactly the same role in our general framework, thereby unifying and extending the irrepresentability condition to a broad class of penalized estimators. We also show that the noiseless recovery condition can be relaxed when turning to so-called thresholded penalized estimators: we prove that the accessibility condition is already sufficient (and necessary) for sure pattern recovery by thresholded penalized estimation provided that the signal of the pattern is large enough. Throughout the article, we demonstrate how our findings can be interpreted through a geometrical lens.