block coordinate descent

This work addresses discrete-time nonlinear optimal control problems by unifying classical algorithms—including gradient descent, Gauss–Newton, Newton’s method, and differential dynamic programming (DDP)—within a differentiable programming framework. Methodologically, it introduces the first modular, end-to-end differentiable algorithm template library built upon linear/quadratic approximations (e.g., LQR), enabled by automatic differentiation. Theoretically, it provides a unified derivation of computational complexity and sufficient optimality conditions across all methods. Practically, it incorporates adaptive line search and regularization strategies, and validates efficacy on benchmark tasks such as autonomous racing with a bicycle model. All implementations are open-sourced, demonstrating both efficient gradient propagation and strong generalization across diverse control problems.

For real-time parametric optimization problems (e.g., model predictive control), this paper proposes an end-to-end self-supervised neural iterative solver: a neural network first generates high-quality initial points, which are then refined by a differentiable primal-dual iterative module. The key contributions are twofold: (i) the design of the first KKT-based, label-free loss function, whose global minima are theoretically guaranteed to coincide exactly with KKT points; and (ii) a local convexification approximation strategy for non-convex problems, extending convergence guarantees to non-convex settings. The method requires no ground-truth labels and enables purely self-supervised training. Evaluated on two canonical non-convex benchmark tasks, it achieves a 10× speedup over IPOPT while attaining solution accuracy orders of magnitude higher than existing learning-based approaches.

This work systematically uncovers the decisive role of problem geometry—specifically, the curvature of the constraint set and the structure of gradients—in governing the statistical-computational trade-offs of stochastic and online optimization algorithms. We introduce the first geometric measure quantifying the deviation of a constraint set from quadratic convexity, rigorously identifying the geometric origins of suboptimality in subgradient methods. We prove that diagonal-preconditioned SGD achieves minimax-optimal convergence rates under quadratic convex constraints. For non-Euclidean, non-quadratically-convex domains—such as ℓₚ-balls with p < 2—we establish tight convergence bounds for mirror descent and adaptive gradient methods, and uncover, for the first time, a precise correspondence between their convergence rates and the accuracy-computation trade-off in Gaussian sequence estimation. Our results provide geometric criteria for algorithm selection and unify the understanding of when nonlinear updates—e.g., via mirror descent—are necessary to attain statistical optimality.

This work addresses large-scale nonlinear equation (root-finding) problems. We propose two stochastic block-coordinate optimistic gradient algorithms—non-accelerated and accelerated versions—guaranteeing convergence under weak Minty solution and co-coercivity assumptions, respectively. Our contributions are threefold: (i) the first integration of the optimistic gradient mechanism with stochastic block-coordinate updates; (ii) the first accelerated block-coordinate root-finding algorithm with almost-sure convergence guarantees; and (iii) an extension to finite-sum inclusion problems, yielding a novel federated learning solver. Theoretically, we establish optimal iteration complexities of $O(1/k)$ for the non-accelerated variant and $O(1/k^2)$ for the accelerated one, along with almost-sure convergence and an almost-sure convergence rate of $O(1/sqrt{k})$. Extensive experiments on synthetic and real-world datasets demonstrate that our methods significantly outperform existing state-of-the-art algorithms.

Conventional design of convex optimization algorithms is often ad hoc and lacks systematic principles. Method: This paper proposes a novel algorithm construction paradigm grounded in RLC circuit modeling: (i) formulate a continuous-time circuit dynamical system whose trajectories converge to the optimizer; (ii) apply automated symbolic discretization coupled with Lyapunov stability analysis to rigorously guarantee global convergence of the resulting discrete-time iterative algorithm. Contribution/Results: This work establishes the first systematic mapping from circuit physics to optimization algorithm design, enabling provably convergent translation from continuous dynamics to discrete algorithms. It uniformly reconstructs classical methods—including gradient descent and Nesterov’s accelerated gradient—and synthesizes multiple new variants, including distributed algorithms. All derived algorithms come with formal convergence proofs, demonstrating the framework’s generality, mathematical rigor, and practical applicability.

This work addresses the computational inefficiency in solving mixed-integer convex optimization problems involving binary indicator variables that govern continuous variables. To tackle this challenge, the authors propose the Coordinate Optimality Reconstruction (CORe) framework, which uniquely integrates coordinate-wise optimality conditions into the modeling of indicator variables. By combining closed-form characterizations with disjunctive reformulation techniques, CORe constructs a novel mixed-integer convex programming formulation that effectively exploits exploitable structures embedded in the problem’s sparsity pattern. The approach preserves global optimality while substantially enhancing the performance of branch-and-bound algorithms. Experimental results demonstrate that, across multiple problem classes—including quadratic programs and robust single-index models—CORe significantly accelerates solver convergence compared to conventional big-M formulations.

This work addresses the computational intractability of large-scale combinatorial optimization problems arising from their exponentially sized search spaces by proposing a structure-aware parallel decomposition framework. The approach constructs a constrained maximum-cut model based on variable interaction structures, reformulates it as a QUBO problem, and leverages an Ising machine to efficiently cluster variables for automatic problem decomposition. The resulting subproblems are then solved in parallel using mathematical optimization solvers. This method uniquely integrates structure-aware clustering with Ising-based computation, substantially reducing the effective problem size. Experimental results on the capacitated vehicle routing problem demonstrate up to a 95.32% reduction in variable count, achieving within one minute the solution quality that conventional methods require thirty minutes to attain, while significantly improving the rate of feasible solutions.

This study investigates the differences in convergence and stability between sequential and parallel coordinate ascent variational inference (CAVI) in moderate-to-high-dimensional linear regression models. By integrating numerical analysis with optimization theory, the work systematically compares the convergence behavior of these two algorithmic variants. The analysis reveals that sequential CAVI enjoys convergence guarantees under substantially milder conditions, whereas parallel CAVI, despite its superior computational efficiency, requires stricter assumptions on the model structure. This research addresses a notable gap in the theoretical understanding of how update strategies in variational inference affect convergence criteria, thereby providing a principled foundation for selecting appropriate algorithms in practical applications.

Existing discrete optimization benchmarks lack fine-grained control over problem characteristics, limiting in-depth analysis of algorithmic behavior. This work proposes a modular benchmark construction framework based on block functions, configurable weights, and dependency graphs, enabling explicit manipulation of problem structure—such as objective space morphology and variable interdependencies—for the first time. The approach facilitates tracking algorithm dynamics at both objective and variable levels and has been successfully applied to analyze the behavior of large-scale, multimodal discrete heuristic algorithms. By offering precise control over problem properties, this framework establishes a new paradigm for research into adaptive mechanisms, diversity management, and dynamic or multi-objective optimization.

This work addresses the lack of systematic methodologies in model optimization, which often relies on heuristic choices and struggles to accommodate diverse deployment constraints. It formalizes model compression and acceleration as a constraint-aware multi-objective engineering decision problem, establishing a unified and actionable framework grounded in five key dimensions: data availability, latency, memory footprint, accuracy tolerance, and retraining budget. By integrating techniques such as quantization, pruning, knowledge distillation, parameter-efficient fine-tuning (PEFT), and inference optimization, the study proposes tailored optimization pipelines for four representative industrial scenarios, delivering a reproducible and quantifiable guide for technology selection.