iterative joint optimization

This work investigates the stability of primal-dual gradient flow dynamics for multi-block composite convex optimization under generalized consensus constraints, particularly targeting large-scale distributed settings involving multiple nonsmooth terms. We propose a continuous-time dynamical framework based on the proximal augmented Lagrangian and establish global exponential convergence via Lyapunov analysis. Compared with mainstream discrete-time algorithms such as ADMM and EXTRA, our approach significantly relaxes standard assumptions—namely, strong convexity and smoothness of objective components, as well as algebraic connectivity of the communication graph—and further proves the necessity of certain relaxed conditions. The theoretical results provide milder, more broadly applicable convergence guarantees for distributed nonsmooth optimization. Numerical experiments demonstrate the efficiency and practicality of the proposed dynamics in both parallel and distributed implementations.

Weighted Low-Rank Approximation (WLRA) seeks a rank-$k$ matrix $XY^ op$ minimizing the weighted Frobenius norm $|W circ (M - XY^ op)|_F$, given a matrix $M$ and a nonnegative weight matrix $W$. This problem is NP-hard and hard to approximate. This paper proposes the first alternating minimization framework for WLRA that simultaneously achieves strong theoretical guarantees and high efficiency. Our method integrates a high-accuracy multi-response regression solver into each alternating update step, enabling approximate yet controllable subproblem solving. Crucially, it preserves global convergence while reducing the per-iteration time complexity from $O(|W|_0 k^2)$ to $O(|W|_0 k)$, where $|W|_0$ denotes the number of nonzero entries in $W$—yielding substantial speedups for sparse weighting patterns. Experiments demonstrate state-of-the-art performance on matrix completion and noise-robust recovery tasks.

This work addresses large-scale nonlinear equation (root-finding) problems. We propose two stochastic block-coordinate optimistic gradient algorithms—non-accelerated and accelerated versions—guaranteeing convergence under weak Minty solution and co-coercivity assumptions, respectively. Our contributions are threefold: (i) the first integration of the optimistic gradient mechanism with stochastic block-coordinate updates; (ii) the first accelerated block-coordinate root-finding algorithm with almost-sure convergence guarantees; and (iii) an extension to finite-sum inclusion problems, yielding a novel federated learning solver. Theoretically, we establish optimal iteration complexities of $O(1/k)$ for the non-accelerated variant and $O(1/k^2)$ for the accelerated one, along with almost-sure convergence and an almost-sure convergence rate of $O(1/sqrt{k})$. Extensive experiments on synthetic and real-world datasets demonstrate that our methods significantly outperform existing state-of-the-art algorithms.

For nonconvex optimization problems with nonlinear equality constraints, this paper proposes an inexact augmented Lagrangian method employing a norm penalty with exponent strictly between 1 and 2. The method constructs Hölder-smooth subproblems under convex feasibility and weak regularity assumptions, leveraging the first use of a non-integer-power Euclidean norm as the augmentation term. We establish, for the first time, accelerated first-order algorithm complexity bounds for such subproblems. Theoretically, we reveal an intrinsic trade-off: constraint violation converges faster as the exponent decreases, while dual residual decay remains controllably degraded. Numerical experiments demonstrate that the proposed method achieves superior constraint satisfaction accuracy and iteration efficiency compared to the standard squared-augmented Lagrangian method.

This paper addresses first-order optimization under nonlinear constraints—including nonconvex feasible sets—by proposing a novel accelerated algorithm grounded in nonsmooth dynamical systems. Methodologically, it models constraints in the **velocity space**, rather than the conventional position space, yielding sparse, local, and convex approximations of the feasible set and eliminating the need for expensive global projections at each iteration. Theoretically, the algorithm converges to stable points under nonconvex objectives and nonconvex constraints; under convexity, it achieves optimal acceleration rates in both continuous- and discrete-time settings. Its computational complexity scales nearly linearly with problem dimension and constraint count. Empirically, the method efficiently solves ℓ^p (p < 1) nonconvex regularized problems in compressed sensing and sparse regression: at p = 1, it matches state-of-the-art performance and substantially outperforms existing approaches.

This work addresses optimization problems with convex constraints whose intersection is difficult to project onto, covering both strongly convex smooth and general nonsmooth convex settings. The authors propose a novel algorithm that integrates stochastic feasibility methods with (sub)gradient descent, wherein each iteration randomly samples a subset of constraints and employs an adaptive Polyak stepsize that requires no prior knowledge of problem parameters, complemented by iterate averaging. Theoretical analysis establishes linear convergence under strong convexity and a worst-case rate of $O(1/\sqrt{T})$ for general convex objectives, while the infeasibility measure decays geometrically almost surely. Numerical experiments on QCQP and SVM tasks demonstrate superior computational efficiency over existing methods, and under specific sampling strategies, the algorithm achieves optimal convergence rates.

This work addresses the computational inefficiency in solving mixed-integer convex optimization problems involving binary indicator variables that govern continuous variables. To tackle this challenge, the authors propose the Coordinate Optimality Reconstruction (CORe) framework, which uniquely integrates coordinate-wise optimality conditions into the modeling of indicator variables. By combining closed-form characterizations with disjunctive reformulation techniques, CORe constructs a novel mixed-integer convex programming formulation that effectively exploits exploitable structures embedded in the problem’s sparsity pattern. The approach preserves global optimality while substantially enhancing the performance of branch-and-bound algorithms. Experimental results demonstrate that, across multiple problem classes—including quadratic programs and robust single-index models—CORe significantly accelerates solver convergence compared to conventional big-M formulations.

This work addresses nonconvex equality-constrained optimization by proposing a gradient-eigenstep algorithm based on the Fletcher augmented Lagrangian function to efficiently compute approximate second-order stationary points. Under suitable initialization and parameter conditions, the algorithm is shown for the first time to enjoy local linear convergence in a neighborhood of strong second-order stationary points. Furthermore, when embedded as a subproblem solver within an incremental sampling strategy, the method significantly outperforms approaches that directly solve the full-sample problem for large-scale stochastic constrained optimization, thereby substantially reducing worst-case sample complexity.

This work addresses the lack of block coordinate descent (BCD) methods with convergence guarantees for nonconvex composite optimization by proposing a unified BCD framework that, for the first time, provides general convergence assurances for such problems. The framework accommodates a variety of mainstream update strategies—including variable-metric proximal gradient, proximal Newton, and alternating minimization—and naturally encompasses Graphical Lasso-type algorithms such as ISTA, Primal GLasso, and QUIC. When applied to sparse precision matrix estimation, the proposed method achieves state-of-the-art estimation accuracy while delivering up to 100-fold acceleration in iteration speed, substantially enhancing computational efficiency.

This work addresses the computational intractability of large-scale combinatorial optimization problems arising from their exponentially sized search spaces by proposing a structure-aware parallel decomposition framework. The approach constructs a constrained maximum-cut model based on variable interaction structures, reformulates it as a QUBO problem, and leverages an Ising machine to efficiently cluster variables for automatic problem decomposition. The resulting subproblems are then solved in parallel using mathematical optimization solvers. This method uniquely integrates structure-aware clustering with Ising-based computation, substantially reducing the effective problem size. Experimental results on the capacitated vehicle routing problem demonstrate up to a 95.32% reduction in variable count, achieving within one minute the solution quality that conventional methods require thirty minutes to attain, while significantly improving the rate of feasible solutions.