apply graph-based regularization

This paper addresses parameter estimation in statistical models with nonconvex regularizers—such as ℓ₀, SCAD, and MCP—by proposing the first decision diagram–based global optimization framework operating directly in the original variable space. The method constructs a compact convex relaxation via a graph-structured representation and integrates it into a spatial branch-and-bound scheme, eliminating the need for auxiliary binary variables or artificial variable bounds. It unifies treatment of generalized ℓₚ norms (p ∈ [0, ∞)) and intricate nonconvex penalty functions. Compared to existing approaches, it achieves superior generalizability and tighter relaxations while guaranteeing finite-time convergence to the global optimum under mild assumptions. Empirically, on sparse linear regression benchmarks, it is the first method to successfully solve SCAD- and MCP-regularized instances that are intractable for conventional global optimizers—demonstrating feasibility, numerical accuracy, and theoretical soundness.

Regularization is often used in high-dimensional regression settings to generate a sparse model, which can save tremendous computing resources and identify predictors that are most strongly associated with the response. When the predictors can be represented by a Gaussian graphical model, the structure of the predictor graph can be exploited during regularization. Our proposed model exploits this underlying predictor graph structure by decomposing the estimated coefficient vector into a sum of latent variables that correspond to the sum of each node contribution to the coefficient vector. Regularization is then performed on the latent variables rather than on the coefficient vector directly. We use a penalty function that permits a clear user-defined trade-off between the L1 and L2 penalties and propose a novel proximal projection during optimization. Further, our implementation computes the projection operator for the intersection of selected groups, which conserves more computing resources compared to predictor duplication methods, especially for high-dimensional data. Through simulation, we evaluate the performance of our approach under different graph structures and node counts, and present results on real-world data. Results suggest that our method exhibits stable performance relative to other singly or doubly sparse graphical regression models.

This paper addresses distributed parameter estimation over multi-task graphs, where agents pursue heterogeneous objectives yet share latent structural relationships across tasks. To enable collaborative learning, such inter-task dependencies are modeled via nonsmooth regularization—overcoming the limitation of conventional smooth regularizers in capturing piecewise-constant parameter jumps. We propose a decentralized learning framework incorporating nonsmooth penalties (e.g., ℓ₀/ℓ₁, elastic net) and employ a forward-backward splitting strategy for efficient distributed optimization. Theoretically, the algorithm is proven to converge in mean-square error to an O(μ)-optimal solution; closed-form analysis is provided even under nonconvex regularizations. Extensive simulations validate its effectiveness and robustness in modeling sparsity and piecewise-constant structures. The key contribution lies in the first systematic integration of nonsmooth regularization into multi-task graph learning, thereby relaxing restrictive smoothness assumptions and enabling more realistic, expressive modeling of task relationships.

This work addresses ill-posed inverse problems governed by partial differential equations (PDEs). We propose a physics-informed, data-driven iterative regularization method that integrates physical modeling with deep learning. Specifically, we construct a graph structure via finite-element discretization, embed the forward operator into a graph neural network (GNN) framework, and design a physically interpretable GNN-based regularizer. During iteration, coefficient reconstruction and the regularization prior are jointly optimized. Unlike conventional Tikhonov or total variation regularization, our approach achieves robustness, generality, and interpretability without requiring strong prior assumptions. Experiments demonstrate significantly improved reconstruction accuracy over classical methods under highly ill-conditioned settings and low signal-to-noise ratios. The method establishes a novel paradigm for synergistic data–physics integration in solving PDE-constrained inverse problems.

This paper addresses graph-structured matrix completion, aiming to overcome three key limitations of conventional graph Laplacian regularization: (i) modeling only local similarity, (ii) sensitivity to spurious edges, and (iii) lack of statistical and computational complexity guarantees. To this end, we propose GSGD—a nonconvex optimization algorithm based on preconditioned projected gradient descent—incorporating higher-order spectral graph propagation to capture long-range dependencies among variables. We establish, for the first time under a nonconvex setting, theoretical guarantees of linear convergence rate and near-optimal sample complexity, while significantly enhancing robustness to graph noise. Experiments on both synthetic and real-world datasets demonstrate superior recovery accuracy and scalability, surpassing the performance ceiling of classical graph-regularized paradigms.

Conventional curvature-based regularization methods struggle to simultaneously preserve sharp edges and maintain isotropic fidelity in surface and image smoothing. Method: This paper proposes a total curvature regularization model incorporating multi-directional normal curvature penalization. We introduce a novel total normal curvature regularizer that adaptively weights principal curvature directions, enabling isotropic modeling for the first time. The resulting high-order nonconvex optimization problem is reformulated as a steady-state partial differential equation (PDE), solved efficiently via operator-splitting time discretization and closed-form subproblem solvers. Contribution/Results: Experiments demonstrate superior robustness across diverse noise types and geometric structures, markedly reduced parameter sensitivity, and significant improvements in both edge preservation accuracy and isotropic fidelity compared to classical curvature regularization approaches.

This work addresses the computational expense and ill-conditioning of graph Laplacian pseudoinverse computation, which hinders scalability in large-scale graph learning. To overcome this, the authors propose a Difference-of-Convex Regularization (DCR) framework that approximates the spectral action of the pseudoinverse through regularized maximum likelihood estimation, thereby avoiding explicit matrix inversion. By introducing a dual representation, the method decouples pseudoinverse learning from instance-level inference, enabling efficient primal problem reconstruction. This study pioneers the integration of difference-of-convex optimization with differentiable programming in graph learning and provides theoretical guarantees on algorithmic convergence and the existence of a unique fixed point. Extensive experiments demonstrate that the proposed approach consistently outperforms conventional convex solvers and graph filtering baselines across diverse graph topologies.

This study addresses the computational challenges posed by high-dimensional regularized estimating equations, which often exhibit non-gradient structures, asymmetric Jacobians, over-identification, non-smoothness, non-convexity, or nested optimization, rendering standard penalized methods inefficient. To tackle this, the paper proposes a unified formulation of such problems as fixed-point equations and systematically develops four computational paradigms—minimization-based, Dantzig-type, regularization-based, and fixed-point-based—integrating strategies from penalized optimization, constrained linear programming, iterative root-finding, and proximal fixed-point iterations. This cohesive framework substantially enhances both solvability and algorithmic stability for high-dimensional regularized estimating equations, demonstrating broad applicability to complex settings such as longitudinal data analysis and survival modeling.

The rule of thumb regarding the relationship between the bias-variance tradeoff and model size plays a key role in classical machine learning, but is now well-known to break down in the overparameterized setting as per the double descent curve. In particular, minimum-norm interpolating estimators can perform well, suggesting the need for new tradeoff in these settings. Accordingly, we propose a regularization-sharpness tradeoff for overparameterized linear regression with an $\ell^p$ penalty. Inspired by the interpolating information criterion, our framework decomposes the selection penalty into a regularization term (quantifying the alignment of the regularizer and the interpolator) and a geometric sharpness term on the interpolating manifold (quantifying the effect of local perturbations), yielding a tradeoff analogous to bias-variance. Building on prior analyses that established this information criterion for ridge regularizers, this work first provides a general expression of the interpolating information criterion for $\ell^p$ regularizers where $p \ge 2$. Subsequently, we extend this to the LASSO interpolator with $\ell^1$ regularizer, which induces stronger sparsity. Empirical results on real-world datasets with random Fourier features and polynomials validate our theory, demonstrating how the tradeoff terms can distinguish performant linear interpolators from weaker ones.

This work addresses the tendency of Laplacian-constrained graphical models—such as the Laplacian-constrained Gaussian graphical model (LCGGM) and the Hüsler–Reiss model—to yield overly dense graphs during structure learning, which compromises interpretability and scalability. The paper introduces, for the first time, spectral graph sparsification as a post-processing step: without requiring additional hyperparameter tuning, it replaces the original Laplacian estimate with a spectrally approximated sparse Laplacian and refits the model. This approach effectively enhances both the sparsity and accuracy of the estimated graph structure. Integrating spectral graph theory, Laplacian-constrained Gaussian graphical models, extreme-value graphical models, and graph sparsification techniques, the method demonstrates superior performance on Erdős–Rényi and stochastic block model simulations and validates its practical utility on real-world data.