apply regularization techniques

This work addresses the efficient computation of L1 regularization paths in linear models, encompassing applications such as LASSO, linear support vector machines, and L1-regularized Kalman smoothing. The authors propose a factor graph approach based on parametric Gaussian message passing, which employs forward–backward recursions to separately handle L1 penalties on predictors and responses, yielding a pair of dual algorithms. This is the first method to integrate parametric Gaussian message passing into L1 path computation, substantially extending sparse modeling capabilities within a state-space framework. The algorithm is highly general, relying primarily on matrix multiplications, and achieves computational complexity that improves upon existing approaches in certain regimes.

本文通过介绍数值线性代数在偏微分方程、机器学习和数据同化中的应用，展示了如何使用少量核心概念解决大规模稀疏系统问题。

In large-scale Gaussian process (GP) hyperparameter optimization, iterative linear solvers—such as conjugate gradient (CG)—induce inefficiency in computing gradients of the marginal likelihood due to repeated, costly matrix-vector operations. Method: We propose a general-purpose optimization framework integrating pathwise gradient estimation, solver warm-starting, and budget-aware early stopping. The framework is agnostic to the underlying iterative solver and supports CG, alternating projections, and stochastic gradient descent. Contribution/Results: Our approach substantially alleviates the accuracy–efficiency trade-off in gradient estimation. Experiments demonstrate up to 72× speedup over standard CG when solving to full convergence. Under early stopping, the average residual norm drops to one-seventh of that achieved by baseline methods, significantly shortening hyperparameter optimization time while preserving convergence stability and gradient estimation accuracy.

This paper addresses the finite-sample identification of the system matrix (A^*) for linear dynamical systems under convex set constraints, based on a single trajectory of length (T). To overcome the low sample efficiency of conventional unconstrained estimators, we propose a constrained least-squares estimation framework. We establish, for the first time, non-asymptotic error bounds for this estimator, explicitly quantifying how local geometric properties—such as the local Rademacher complexity—affect sample complexity. Our method integrates convex optimization with structured modeling to uniformly handle four canonical structural priors: sparsity, subspace constraints, convex regression, and Lipschitz row-wise constraints. Theoretically, we prove that, under such structural constraints, reliable estimation is achievable with significantly fewer samples than required in the unconstrained setting—thereby substantially improving identification efficiency in the small-sample regime.

This work addresses the convergence guarantees of stochastic line search optimization for over-parameterized models under interpolation conditions. We establish a necessary and sufficient condition on the search direction—applicable to a broad class of methods—that ensures finite termination and bounded backtracking steps, and rigorously prove linear convergence under the Polyak–Łojasiewicz (PL) assumption. The condition unifies major first-order strategies—including momentum, conjugate gradient, and adaptive preconditioning—providing a verifiable theoretical foundation for their principled integration with stochastic line search. Our analysis fills a critical gap in the convergence theory of stochastic line search methods and significantly extends both the applicability and reliability of efficient first-order optimization in interpolation learning regimes.

Real-time, sparse identification of dynamic models in partially observable systems remains challenging due to limited observability and computational constraints. Method: This paper proposes an online sparse Kalman identification method that integrates a Bayesian sparsification mechanism—based on Automatic Relevance Determination (ARD)—into the Augmented Kalman Filter (AKF) framework. It establishes an adaptive posterior update scheme enabling the basis function set to evolve dynamically with incoming observations, and derives explicit gradient-descent update rules to accelerate sparse learning. Contribution/Results: Unlike conventional batch-mode sparse identification methods, the proposed approach enables incremental modeling over sequential data, significantly improving real-time capability and interpretability. Experimental results demonstrate an 84.21% improvement in model accuracy over standard AKF under millisecond-level computational latency, validating its effectiveness and robustness in both simulated and physical systems.

This study addresses the computational challenges posed by high-dimensional regularized estimating equations, which often exhibit non-gradient structures, asymmetric Jacobians, over-identification, non-smoothness, non-convexity, or nested optimization, rendering standard penalized methods inefficient. To tackle this, the paper proposes a unified formulation of such problems as fixed-point equations and systematically develops four computational paradigms—minimization-based, Dantzig-type, regularization-based, and fixed-point-based—integrating strategies from penalized optimization, constrained linear programming, iterative root-finding, and proximal fixed-point iterations. This cohesive framework substantially enhances both solvability and algorithmic stability for high-dimensional regularized estimating equations, demonstrating broad applicability to complex settings such as longitudinal data analysis and survival modeling.

Robust sparse estimation of high-dimensional covariance matrices faces three interrelated challenges: difficulty in guaranteeing positive definiteness, sparsity degradation due to post-hoc corrections, and uncontrolled condition numbers. Method: We propose the first method that explicitly incorporates a condition-number constraint into a robust adaptive thresholding framework. Using convex optimization and a provably convergent alternating direction algorithm, our approach jointly ensures positive definiteness, sparsity, and numerical stability. Contribution/Results: We establish theoretical minimax optimal convergence rate under the Frobenius norm. Experiments on both synthetic and real-world datasets demonstrate that our estimator consistently yields positive definite, sparse, and well-conditioned (low condition number) covariance matrices. Its numerical stability matches or surpasses that of eigenvalue truncation, while requiring fewer hyperparameters and offering greater practical utility.

This study addresses the existence, uniqueness, and algorithmic convergence of covariance matrix estimators in penalized multivariate divergence-based M-estimation under structural constraints. By constructing a regularized M-estimation framework with geodesically convex (not necessarily smooth) penalty functions, the authors rigorously establish the well-posedness of the resulting estimator. They further demonstrate, for the first time, that the standard fixed-point algorithm may fail in this setting and propose a novel reweighted algorithm that integrates tools from geodesic convex analysis and nonsmooth optimization. This new method guarantees monotone convergence for a broad class of penalized M-estimation problems involving structured covariance matrices.

This work addresses the challenge that existing data-driven linear parameter-varying (LPV) modeling approaches struggle to quantify model uncertainty, thereby hindering reliable assessment of prediction credibility or detection of out-of-distribution operating conditions. The paper proposes a Bayesian framework that, for the first time in LPV modeling, jointly accounts for aleatoric uncertainty arising from measurement noise and epistemic uncertainty stemming from limited data and structural bias. Within this framework, both the LPV state-space model and its scheduling variables are estimated simultaneously, yielding predictive confidence intervals. The approach preserves the standard LPV structure, ensuring compatibility with subsequent controller synthesis, and demonstrates high-fidelity modeling capability and robust uncertainty quantification on a two-dimensional nonlinear mass–spring–damper interconnected system.