constrained optimization

This work addresses integer and mixed-integer nonlinear programming (INLP/MINLP) problems, aiming to accelerate exact algorithms—particularly branch-and-bound (BB)—while rigorously preserving global optimality. We propose a unified learnable BB framework that jointly integrates supervised learning, imitation learning, and reinforcement learning into four core components: branching variable selection, cutting-plane generation, node prioritization, and parameter tuning. The framework is agnostic to variable types—supporting discrete, continuous, and hybrid structures—and is validated on real-world applications including unit commitment, vehicle routing, and hydroelectric scheduling. Key contributions include: (i) the first taxonomy of learning-augmented optimization methods organized along both solver architecture and learning paradigm dimensions; (ii) substantial convergence acceleration without compromising solution quality or optimality guarantees; and (iii) advancement toward scalable, generalizable intelligent optimization solvers.

In combinatorial optimization, the empirical risk w.r.t. model parameters is piecewise constant, hindering gradient-based optimization and lacking theoretical generalization guarantees. Method: For contextual stochastic optimization with complex objectives, we propose a perturbation-driven risk smoothing strategy. Our approach integrates statistical learning models with a surrogate combinatorial optimization oracle to construct a context-aware, generalization-controllable decision framework. Contribution/Results: We establish the first unified generalization bound incorporating perturbation bias, statistical error, and optimization error. We introduce the notion of “uniform weak consistency” to characterize the coupled stability between the learning model and the surrogate oracle, proving its universality under mild assumptions. Experiments on stochastic vehicle scheduling demonstrate strong generalization performance. This work provides the first verifiable theoretical generalization framework for contextual stochastic optimization.

Linearly convergent optimization algorithms often exhibit suboptimal average-case performance on practical problem instances, yet their worst-case convergence guarantees—critical for safety-critical applications—must be preserved. Method: We propose a safety-enhancing framework grounded in nonsmooth composite optimization, which, for the first time, fully characterizes the class of admissible update-rule transformations that preserve linear convergence. This unified characterization encompasses foundational algorithms—including gradient descent, Nesterov’s accelerated method, and projection-based methods—as special cases. Contribution/Results: Our framework strictly maintains theoretical linear convergence rates while significantly accelerating empirical convergence in demanding settings such as ill-conditioned linear systems solving and model predictive control. Experiments demonstrate substantial improvements in average solution efficiency under finite iteration budgets, achieving both rigorous worst-case guarantees and strong practical performance.

This work systematically uncovers the decisive role of problem geometry—specifically, the curvature of the constraint set and the structure of gradients—in governing the statistical-computational trade-offs of stochastic and online optimization algorithms. We introduce the first geometric measure quantifying the deviation of a constraint set from quadratic convexity, rigorously identifying the geometric origins of suboptimality in subgradient methods. We prove that diagonal-preconditioned SGD achieves minimax-optimal convergence rates under quadratic convex constraints. For non-Euclidean, non-quadratically-convex domains—such as ℓₚ-balls with p < 2—we establish tight convergence bounds for mirror descent and adaptive gradient methods, and uncover, for the first time, a precise correspondence between their convergence rates and the accuracy-computation trade-off in Gaussian sequence estimation. Our results provide geometric criteria for algorithm selection and unify the understanding of when nonlinear updates—e.g., via mirror descent—are necessary to attain statistical optimality.

This work addresses the convergence guarantees of stochastic line search optimization for over-parameterized models under interpolation conditions. We establish a necessary and sufficient condition on the search direction—applicable to a broad class of methods—that ensures finite termination and bounded backtracking steps, and rigorously prove linear convergence under the Polyak–Łojasiewicz (PL) assumption. The condition unifies major first-order strategies—including momentum, conjugate gradient, and adaptive preconditioning—providing a verifiable theoretical foundation for their principled integration with stochastic line search. Our analysis fills a critical gap in the convergence theory of stochastic line search methods and significantly extends both the applicability and reliability of efficient first-order optimization in interpolation learning regimes.

This work addresses constraint satisfaction problems involving random variables, aiming to find deterministic parameters that maximize the probability of satisfying all constraints under uncertainty. To this end, the authors propose a novel approach that integrates oracle-based stochastic gradient descent with interval arithmetic: the former efficiently explores high-quality parameter configurations, while the latter provides rigorous and successively tighter lower bounds on the satisfaction probability. This is the first method to synergistically combine these two techniques for solving stochastic constraints, offering both high-probability convergence guarantees and computational reliability. Experimental results on stochastic satisfiability modulo theories (SSMT) and stochastic trajectory planning tasks demonstrate that the proposed method efficiently generates sequences of reliable lower bounds that closely approximate the true optimal value.

This work addresses the issue that quadratic penalty relaxations of binary linear programs often yield spurious or infeasible local minima. To overcome this, we propose a class of QUBO relaxation models satisfying specific structural conditions that guarantee all local minima are feasible and strictly binary. Leveraging these conditions, we derive novel differentiable relaxations for classical combinatorial optimization problems—including open-pit mining, the 0–1 knapsack problem, and the traveling salesman problem—and solve them using gradient-based optimizers such as projected gradient descent and Adam. Experimental results demonstrate that the proposed approach reliably converges to valid binary solutions, thereby establishing clear theoretical guarantees and delineating the applicability boundaries of differentiable optimization as a local solver for combinatorial problems.

This work addresses the lack of systematic methodologies in model optimization, which often relies on heuristic choices and struggles to accommodate diverse deployment constraints. It formalizes model compression and acceleration as a constraint-aware multi-objective engineering decision problem, establishing a unified and actionable framework grounded in five key dimensions: data availability, latency, memory footprint, accuracy tolerance, and retraining budget. By integrating techniques such as quantization, pruning, knowledge distillation, parameter-efficient fine-tuning (PEFT), and inference optimization, the study proposes tailored optimization pipelines for four representative industrial scenarios, delivering a reproducible and quantifiable guide for technology selection.

This study investigates the statistical properties of Lagrange multipliers in constrained maximum likelihood estimation and least squares problems, along with their implications for numerical optimization. Leveraging large-sample theory, it establishes that under correctly specified models, Lagrange multipliers converge in probability to zero as the sample size grows, a result extended to high-dimensional settings such as deep learning. Building on this asymptotic behavior, the work provides the first statistical justification for initializing Lagrange multipliers at zero and integrates this insight into constrained optimization algorithms, including augmented Lagrangian methods and sequential quadratic programming. Numerical experiments demonstrate that this initialization strategy substantially enhances algorithmic stability and convergence efficiency in applications such as constrained regression and dynamic discrete choice models.

To address the challenge of solving parametric nonlinear constrained optimization problems at high frequency in real-time control and model-based design, this paper proposes an end-to-end neural network that directly learns the mapping from problem parameters to both primal and dual variables. Our method innovatively incorporates the Karush–Kuhn–Tucker (KKT) optimality residual into the loss function and employs constraint-aware output activations (e.g., Softplus or Clamp) to intrinsically enforce feasibility and optimality during training—thereby reducing data dependency and enabling high-accuracy dual variable prediction. Experiments demonstrate that, compared to a quadratic penalty baseline, our approach reduces constraint violations by 37%, decreases primal variable error by 22%, achieves dual variable prediction error below 0.05, and exhibits superior hyperparameter robustness.