warm-starting solvers

To address the slow convergence and suboptimal solutions of local optimization methods in real-time control—stemming from reliance on a single, fixed initial solution—this paper proposes a learning-based framework for multi-initial-solution prediction. Methodologically, it formulates diverse initial-solution generation as a supervised learning task for the first time and incorporates meta-learning to enhance cross-task generalization. Two complementary execution strategies are introduced: (i) adaptive selection of a single optimizer and (ii) parallel execution of multiple optimizers—both rigorously guaranteeing that the final solution is no worse than that obtained from default initialization. The framework is compatible with various optimal control optimizers, including DDP, MPPI, and iLQR. Evaluated on cart-pole, reacher, and autonomous driving benchmarks, it significantly improves both convergence speed and solution quality under strict time constraints, while scaling efficiently to larger numbers of initial solutions.

For real-time parametric optimization problems (e.g., model predictive control), this paper proposes an end-to-end self-supervised neural iterative solver: a neural network first generates high-quality initial points, which are then refined by a differentiable primal-dual iterative module. The key contributions are twofold: (i) the design of the first KKT-based, label-free loss function, whose global minima are theoretically guaranteed to coincide exactly with KKT points; and (ii) a local convexification approximation strategy for non-convex problems, extending convergence guarantees to non-convex settings. The method requires no ground-truth labels and enables purely self-supervised training. Evaluated on two canonical non-convex benchmark tasks, it achieves a 10× speedup over IPOPT while attaining solution accuracy orders of magnitude higher than existing learning-based approaches.

This work addresses the family of parametric optimization problems and proposes the first unified, data-driven framework for analyzing the generalization performance of both classical and learned optimizers. Methodologically: (1) it introduces PAC-Bayes theory to the analysis of learned optimizers, deriving verifiable, high-probability generalization upper bounds; (2) it establishes performance bounds for classical optimizers based on empirical convergence rates; and (3) it pioneers a learning paradigm that directly minimizes the PAC-Bayes bound during training. Evaluated on signal processing, control, and meta-learning tasks, the derived bounds are significantly tighter than conventional worst-case guarantees. Moreover, the theoretical generalization guarantees for learned optimizers consistently exceed the empirical performance of their non-learned baselines—thereby unifying theoretical rigor with practical efficacy.

Large-scale parametric optimization problems—such as risk assessment and security-constrained optimal power flow (SCOPF) in power systems—pose significant computational challenges due to stringent real-time requirements, high dimensionality, and strict feasibility constraints. Method: This paper proposes a differentiable, feasible, and theoretically grounded optimization surrogate framework. It integrates differentiable programming, feasibility-preserving repair layers, and self-supervised training—eliminating reliance on labeled data and enabling end-to-end learning. Robustness verification and scalable compression techniques are incorporated to guarantee solution feasibility and bounded objective optimality gaps. Contribution/Results: To the best of our knowledge, this is the first optimization surrogate paradigm that simultaneously ensures differentiability, hard constraint satisfaction, and rigorous theoretical performance guarantees. It overcomes critical bottlenecks of conventional solvers in terms of real-time responsiveness and scalability. Experiments on large-scale power system instances demonstrate up to 100× speedup over state-of-the-art optimization methods, while strictly satisfying all security constraints and maintaining high solution accuracy.

This work addresses the lack of provable joint guarantees on generalization and convergence in learned optimization algorithms. Methodologically: (1) it establishes the first PAC-Bayesian generalization bound for unbounded losses, leveraging exponential-family posterior distributions; (2) it formulates optimizer learning as a tractable one-dimensional global optimization problem—convex or non-convex—whose solution is analytically characterizable; and (3) it integrates stochastic optimization design with rigorous theoretical analysis to explicitly trade off convergence rate against generalization error. Empirically, the learned optimizers achieve order-of-magnitude improvements over state-of-the-art methods across four diverse real-world tasks—including neural architecture search, meta-learning, adversarial training, and federated learning—while all gains are underpinned by formal theoretical guarantees. This constitutes the first learning-to-optimize framework endowed with a provably tight PAC-Bayesian generalization bound and jointly certified convergence–generalization performance.

This work proposes a differentiable programming–based framework for learning adaptive optimization algorithms to address the slow convergence and high per-iteration cost of traditional first-order methods in large-scale optimization. By embedding Fenchel–Rockafellar duality theory into automatic differentiation systems, the framework enables end-to-end training and adaptive refinement of duality-driven iterative schemes such as ADMM and PDHG. Implemented uniformly across major deep learning frameworks—including PyTorch, TensorFlow, and JAX—the approach significantly improves both computational efficiency and solution quality on a range of tasks, including linear programming, optimal power flow (OPF), Laplacian regularization, and neural network verification.

High-fidelity partial differential equation (PDE) simulations remain computationally prohibitive for querying, real-time control, and design optimization; while data-driven surrogate models offer speed, they suffer from poor generalizability and limited reliability. This paper introduces Neural Operator Warm-Starting (NOWS), a method that embeds a learned solution operator into classical Krylov subspace iterative solvers—such as conjugate gradient and GMRES—to provide high-quality initial guesses. NOWS preserves the original solver’s architecture and is agnostic to discretization schemes, supporting finite differences, finite elements, and isogeometric analysis while rigorously maintaining numerical stability and convergence guarantees. Evaluated on multiple benchmark PDE problems, NOWS reduces iteration counts substantially and achieves up to 90% end-to-end runtime reduction. It establishes a critical trade-off between speed and reliability—marking the first systematic integration of neural operators as warm-starting components within traditional iterative solvers, rather than as solver replacements.

This work addresses the critical challenge that modern GPU-accelerated linear programming solvers—such as cuPDLP, which is based on the primal-dual hybrid gradient (PDHG) algorithm—exhibit performance highly sensitive to hyperparameters, yet lack tuning methods with provable generalization guarantees. For the first time, this study establishes structural relationships between hyperparameters and solution trajectories for multiple adaptive techniques in complex first-order LP solvers, including preconditioning, restart strategies, and smoothed weight updates. By integrating convergence analysis of PDHG with a model of structural sensitivity, the authors propose a data-driven hyperparameter learning framework that offers theoretical generalization guarantees under polynomial sample complexity. Experimental results demonstrate that the framework significantly enhances solver efficiency across diverse problem instances.

This paper addresses machine unlearning—the efficient removal of a model’s dependence on specific training samples while preserving performance on the remaining data. We propose a constraint-optimization-based feasible update framework. Our core innovation introduces a parameter masking mechanism to select an updateable subspace, jointly incorporating gradient noise modeling and directional constraints on parameter updates to yield locally feasible solutions satisfying both unlearning objectives and utility preservation. The method operates as a plug-and-play module, enhancing the robustness and accuracy of diverse first-order approximate unlearning algorithms. Experiments on image classification tasks demonstrate that our approach significantly improves unlearning accuracy (average gain of 12.3%) while incurring negligible utility loss—less than 0.5% drop in test accuracy on retained data—validating its effectiveness and practicality.

To address the high computational overhead of quadratic programming (QP) solvers in real-time control—where strict timing constraints are critical—this paper proposes a graph neural network (GNN)-based active-set prediction method for efficient warm-starting of dual active-set QP solvers (DAQP). The approach models each QP instance as a structure-aware bipartite graph, enabling the GNN to encode constraint topology and generalize across problem scales. An active-set classifier is trained via supervised learning to predict the initial active set, which is then integrated with DAQP to refine the initialization. Experiments demonstrate that the method substantially reduces the number of solver iterations, achieves significantly faster convergence than cold-start baselines, matches or exceeds the performance of MLP-based predictors, and exhibits strong out-of-distribution generalization to unseen problem dimensions.