New explanations and inference for least angle regression

2026-02-02

AI Summary

This work addresses the lack of theoretical grounding in Least Angle Regression (LAR) regarding variable importance assessment and stopping criteria. The authors propose a novel framework that interprets the LAR path as a sequence of response mean estimates ordered by descending population correlation parameters. They establish that non-zero population correlation estimates along the LAR path follow independent normal distributions, whereas zero-correlation estimates exhibit a specific non-normal joint structure. Leveraging these insights, the study develops a formal stopping rule and introduces a modified bootstrap procedure to quantify uncertainty in the variable selection order. Through simulations and empirical analyses, the proposed approach demonstrates its effectiveness in delivering interpretable measures of variable contribution and reliable termination criteria for the LAR algorithm.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Efron et al. (2004) introduced least angle regression (LAR) as an algorithm for linear predictions, intended as an alternative to forward selection with connections to penalized regression. However, LAR has remained somewhat of a"black box,"where some basic behavioral properties of LAR output are not well understood, including an appropriate termination point for the algorithm. We provide a novel framework for inference with LAR, which also allows LAR to be understood from new perspectives with several newly developed mathematical properties. The LAR algorithm at a data level can viewed as estimating a population counterpart"path"that organizes a response mean along regressor variables which are ordered according to a decreasing series of population"correlation"parameters; such parameters are shown to have meaningful interpretations for explaining variable contributions whereby zero correlations denote unimportant variables. In the output of LAR, estimates of all non-zero population correlations turn out to have independent normal distributions for use in inference, while estimates of zero-valued population correlations have a certain non-normal joint distribution. These properties help to provide a formal rule for stopping the LAR algorithm. While the standard bootstrap for regression can fail for LAR, a modified bootstrap provides a practical and formally justified tool for interpreting the entrance of variables and quantifying uncertainty in estimation. The LAR inference method is studied through simulation and illustrated with data examples.

Problem

• least angle regression
• inference
• algorithm termination
• black box
• variable selection

Innovation

• least angle regression
• statistical inference
• population correlation path
• modified bootstrap
• algorithm stopping rule