adaptive primal-dual optimization

To address the slow convergence of conventional solvers for large-scale constrained optimization and their inability to meet real-time requirements, this paper proposes a novel machine learning surrogate method that learns dual feasible solutions—thereby overcoming the lack of theoretical guarantees inherent in existing primal-only surrogates. Our approach tightly integrates linear programming structure, convex optimization theory, and dual modeling. Key contributions include: (1) the first smooth, self-supervised dual loss function enabling end-to-end differentiable training; and (2) an implicit-layer-free analytical dual completion strategy that rigorously ensures dual feasibility and enables millisecond-scale inference. Evaluated on large-scale linear optimization benchmarks, our method achieves <1% optimality gap while accelerating solution times by several orders of magnitude over commercial solvers, significantly outperforming unstructured surrogate models.

This work systematically uncovers the decisive role of problem geometry—specifically, the curvature of the constraint set and the structure of gradients—in governing the statistical-computational trade-offs of stochastic and online optimization algorithms. We introduce the first geometric measure quantifying the deviation of a constraint set from quadratic convexity, rigorously identifying the geometric origins of suboptimality in subgradient methods. We prove that diagonal-preconditioned SGD achieves minimax-optimal convergence rates under quadratic convex constraints. For non-Euclidean, non-quadratically-convex domains—such as ℓₚ-balls with p < 2—we establish tight convergence bounds for mirror descent and adaptive gradient methods, and uncover, for the first time, a precise correspondence between their convergence rates and the accuracy-computation trade-off in Gaussian sequence estimation. Our results provide geometric criteria for algorithm selection and unify the understanding of when nonlinear updates—e.g., via mirror descent—are necessary to attain statistical optimality.

This paper addresses strongly convex yet nonsmooth and non-Lipschitz optimization problems. We develop a unified primal–dual theoretical framework that, for the first time, reveals the equivalent dual-averaging representations of the subgradient method, proximal subgradient method, and switching subgradient method. Through a novel *dual-gap convergence analysis*, we establish the first $O(1/T)$ convergence guarantee applicable to this problem class. We derive an optimal stopping criterion and optimality certificate that require no additional computation, and rigorously characterize a controllable convergence boundary—even under early-stage exponential divergence. Our theory accommodates a broad range of step-size choices and accommodates ill-conditioned non-Lipschitz structures. While preserving algorithmic simplicity, our framework substantially extends both the applicability and theoretical depth of subgradient-type methods.

This work investigates the stability of primal-dual gradient flow dynamics for multi-block composite convex optimization under generalized consensus constraints, particularly targeting large-scale distributed settings involving multiple nonsmooth terms. We propose a continuous-time dynamical framework based on the proximal augmented Lagrangian and establish global exponential convergence via Lyapunov analysis. Compared with mainstream discrete-time algorithms such as ADMM and EXTRA, our approach significantly relaxes standard assumptions—namely, strong convexity and smoothness of objective components, as well as algebraic connectivity of the communication graph—and further proves the necessity of certain relaxed conditions. The theoretical results provide milder, more broadly applicable convergence guarantees for distributed nonsmooth optimization. Numerical experiments demonstrate the efficiency and practicality of the proposed dynamics in both parallel and distributed implementations.

For real-time parametric optimization problems (e.g., model predictive control), this paper proposes an end-to-end self-supervised neural iterative solver: a neural network first generates high-quality initial points, which are then refined by a differentiable primal-dual iterative module. The key contributions are twofold: (i) the design of the first KKT-based, label-free loss function, whose global minima are theoretically guaranteed to coincide exactly with KKT points; and (ii) a local convexification approximation strategy for non-convex problems, extending convergence guarantees to non-convex settings. The method requires no ground-truth labels and enables purely self-supervised training. Evaluated on two canonical non-convex benchmark tasks, it achieves a 10× speedup over IPOPT while attaining solution accuracy orders of magnitude higher than existing learning-based approaches.

This work addresses the challenge of simultaneously handling adversarial losses and stochastic or adversarial constraints in online convex optimization without requiring the Slater condition. The authors propose an anytime primal-dual framework that stabilizes the dual dynamics by incorporating an adaptive regularization term in the dual updates. This approach is the first to achieve near-optimal regret and constraint violation bounds without assuming the Slater condition, while also providing high-probability guarantees. Specifically, for stochastic constraints and convex losses, the method attains an expected regret of $O(\sqrt{T})$ and expected cumulative constraint violation of $O(\sqrt{T}\log T)$. In the strongly convex setting, the regret improves to $O(\log T)$, and the framework naturally extends to scenarios with adversarial constraints.

This work addresses the challenge of performance instability in online minimization problems caused by minor perturbations in input instances. It introduces, for the first time, dual linear programming predictions into this domain, proposing a novel framework that integrates online algorithms, duality theory, and machine learning. By leveraging the stability of dual solutions across similar problem instances, the approach enhances both learnability and robustness for metric task systems and layered set cover problems. Theoretical analysis and empirical evaluations on the k-server and parking permit problems demonstrate that the proposed algorithm significantly outperforms traditional methods, achieving tighter competitive ratios and superior practical performance.

This work addresses online convex optimization under convex constraints in safety-critical settings, where feasibility must be maintained at every round. The authors propose a novel algorithm, AdaOGD-PFS, which integrates online gradient descent with Polyak feasibility steps, requiring only a single constraint evaluation and one subgradient query per iteration while guaranteeing perpetual feasibility and computational efficiency. The key contribution lies in establishing a data-dependent regret bound that replaces the worst-case dependence on \( G_f^2 T \) with the observed cumulative gradient norm \( G_T \), and revealing that the non-negative Polyak correction term \( P_T \) can negatively contribute to—thereby improving—the regret bound. An adaptive step-size strategy is further developed. Experiments under ball and half-space constraints demonstrate 38%–43% improvement in regret over baseline methods, confirming the efficacy of the proposed approach.

This work addresses online convex optimization problems where both the loss functions and constraints depend on the learner’s finite history of decisions—arising, for instance, in constrained dynamic control and scheduling with reconfiguration budgets. The study investigates settings with memory and time-varying constraints, proposing an algorithm that simultaneously achieves sublinear regret and sublinear cumulative constraint violation, even without predictions or with only short-term (possibly unreliable) forecasts. The key innovation lies in establishing, for the first time, dual sublinear guarantees in constrained online optimization with memory. By modeling predictive information as delayed feedback, the authors design an adaptive, robust optimistic online learning algorithm that maintains theoretical performance in the absence of predictions and automatically improves as prediction accuracy increases, thereby bridging the theoretical gap between classical constrained online optimization and memory-dependent scenarios.