🤖 AI Summary
This work addresses the efficient computation of L1 regularization paths in linear models, encompassing applications such as LASSO, linear support vector machines, and L1-regularized Kalman smoothing. The authors propose a factor graph approach based on parametric Gaussian message passing, which employs forward–backward recursions to separately handle L1 penalties on predictors and responses, yielding a pair of dual algorithms. This is the first method to integrate parametric Gaussian message passing into L1 path computation, substantially extending sparse modeling capabilities within a state-space framework. The algorithm is highly general, relying primarily on matrix multiplications, and achieves computational complexity that improves upon existing approaches in certain regimes.
📝 Abstract
The paper considers the computation of L1 regularization paths in a state space setting, which includes L1 regularized Kalman smoothing, linear SVM, LASSO, and more. The paper proposes two new algorithms, which are duals of each other; the first algorithm applies to L1 regularization of independent variables while the second applies to L1 regularization of dependent variables. The heart of the proposed algorithms is parametric Gaussian message passing (i.e., Kalman-type forward-backward recursions) in the pertinent factor graphs. The proposed methods are broadly applicable, they (usually) require only matrix multiplications, and their complexity can be competitive with prior methods in some cases.