projected gradient methods

For nonconvex optimization problems with nonlinear equality constraints, this paper proposes an inexact augmented Lagrangian method employing a norm penalty with exponent strictly between 1 and 2. The method constructs Hölder-smooth subproblems under convex feasibility and weak regularity assumptions, leveraging the first use of a non-integer-power Euclidean norm as the augmentation term. We establish, for the first time, accelerated first-order algorithm complexity bounds for such subproblems. Theoretically, we reveal an intrinsic trade-off: constraint violation converges faster as the exponent decreases, while dual residual decay remains controllably degraded. Numerical experiments demonstrate that the proposed method achieves superior constraint satisfaction accuracy and iteration efficiency compared to the standard squared-augmented Lagrangian method.

This paper addresses first-order optimization under nonlinear constraints—including nonconvex feasible sets—by proposing a novel accelerated algorithm grounded in nonsmooth dynamical systems. Methodologically, it models constraints in the **velocity space**, rather than the conventional position space, yielding sparse, local, and convex approximations of the feasible set and eliminating the need for expensive global projections at each iteration. Theoretically, the algorithm converges to stable points under nonconvex objectives and nonconvex constraints; under convexity, it achieves optimal acceleration rates in both continuous- and discrete-time settings. Its computational complexity scales nearly linearly with problem dimension and constraint count. Empirically, the method efficiently solves ℓ^p (p < 1) nonconvex regularized problems in compressed sensing and sparse regression: at p = 1, it matches state-of-the-art performance and substantially outperforms existing approaches.

This paper addresses feasibility and optimization problems over smooth and/or strongly convex constraint sets. To overcome the computational bottleneck of conventional methods—relying on expensive projection or linear optimization oracles—we propose a scalable, accelerated first-order algorithm that requires neither projections nor linear oracles, but only adaptive one-dimensional line searches and normal vector computations. Our key contribution is the first establishment of structural inheritance: the squared Minkowski gauge inherits both smoothness and strong convexity from the underlying constraint set, thereby circumventing analytical difficulties arising from its inherent nonsmoothness and lack of strong convexity. Theoretically, the algorithm achieves an $O(1/T)$ convergence rate under strong convexity, $O(1/T^2)$ under smoothness, and accelerated linear convergence when both properties hold—matching known lower bounds. Moreover, per-iteration complexity is substantially reduced.

This paper studies variational inequality (VI) problems with multiple functional constraints. Conventional first-order methods rely on expensive projection or linear minimization oracles, while existing primal-dual algorithms require prior knowledge of optimal Lagrange multipliers. To address these limitations, we propose a purely primal Constraint Gradient Method (CGM), the first algorithm achieving non-asymptotic convergence without any prior information about Lagrange multipliers. Theoretically, under monotonicity and strong monotonicity assumptions, CGM attains the optimal operator query complexity—matching that of projection-based methods—while solving only lightweight quadratic programming subproblems per iteration, thereby substantially reducing computational overhead. Numerical experiments demonstrate that CGM is both efficient and robust on multi-constrained VI problems, consistently outperforming state-of-the-art primal-dual methods in practice.

This work systematically uncovers the decisive role of problem geometry—specifically, the curvature of the constraint set and the structure of gradients—in governing the statistical-computational trade-offs of stochastic and online optimization algorithms. We introduce the first geometric measure quantifying the deviation of a constraint set from quadratic convexity, rigorously identifying the geometric origins of suboptimality in subgradient methods. We prove that diagonal-preconditioned SGD achieves minimax-optimal convergence rates under quadratic convex constraints. For non-Euclidean, non-quadratically-convex domains—such as ℓₚ-balls with p < 2—we establish tight convergence bounds for mirror descent and adaptive gradient methods, and uncover, for the first time, a precise correspondence between their convergence rates and the accuracy-computation trade-off in Gaussian sequence estimation. Our results provide geometric criteria for algorithm selection and unify the understanding of when nonlinear updates—e.g., via mirror descent—are necessary to attain statistical optimality.

This work addresses nonconvex equality-constrained optimization by proposing a gradient-eigenstep algorithm based on the Fletcher augmented Lagrangian function to efficiently compute approximate second-order stationary points. Under suitable initialization and parameter conditions, the algorithm is shown for the first time to enjoy local linear convergence in a neighborhood of strong second-order stationary points. Furthermore, when embedded as a subproblem solver within an incremental sampling strategy, the method significantly outperforms approaches that directly solve the full-sample problem for large-scale stochastic constrained optimization, thereby substantially reducing worst-case sample complexity.

This work addresses the limitations of the classical Frank-Wolfe method, which relies on a global linear minimization oracle (LMO) and requires bounded feasible sets along with curvature assumptions. The authors propose a Local LMO algorithm that solves a local linear minimization problem over the intersection of a neighborhood around the current iterate and the constraint set, thereby enabling projection-free gradient-type optimization. This approach extends the convergence theory of projected gradient descent to projection-free settings, eliminating the need for boundedness and curvature conditions. It provides a unified framework for convex, strongly convex, nonconvex, and stochastic optimization problems. Under various settings, the algorithm achieves optimal convergence rates matching those of projected gradient descent: sublinear for convex objectives, linear for smooth strongly convex cases, and optimal sublinear rates for nonconvex, stochastic, and nonsmooth scenarios.

This work proposes a trajectory-restricted framework for linear convergence analysis that overcomes the conservatism of traditional first-order methods, whose guarantees often rely on global geometric conditions and worst-case constants. Instead of imposing regularity assumptions globally, our approach requires only local geometric properties—such as restricted Polyak–Łojasiewicz inequalities, error bounds, and quadratic growth—on the subset of the space actually traversed by the algorithm. We establish explicit relationships among the associated constants and show that, for piecewise polyhedral composite problems, once iterates enter a well-conditioned active manifold, convergence is governed by the restricted Hoffman constant of that manifold, yielding an improved effective condition number and faster local convergence. The results demonstrate that linear convergence fundamentally depends on the local geometry encountered along the algorithmic trajectory, rather than on global worst-case scenarios.

This work addresses the challenge of efficiently training generative models to produce high-quality initial solutions under data scarcity. The authors propose a k-neighborhood data collection strategy that enriches the training set by reusing intermediate iterates from a projected gradient descent solver, thereby improving data efficiency without incurring additional solve overhead. For the first time, they derive a generalization error bound based on Rademacher complexity for learning from solver trajectories, revealing how the k-neighborhood structure influences generalization performance. The approach bridges data-driven dynamic adaptive systems (DDDAS) with the emerging global search method GLENS. Theoretical analysis and experiments on box-constrained quadratic programming problems demonstrate that the proposed method significantly enhances both data utilization and optimization loop efficiency.

This work addresses the challenge of nonlinear optimization with mixed equality and inequality constraints in robotic dynamics planning by introducing a novel approach based on “constraint manifolds with corners.” The method reformulates the original problem as an unconstrained optimization over a constrained state space, seamlessly embedding inequality constraints into the manifold structure through differential geometry and manifold optimization techniques. This formulation overcomes the conventional limitation of manifold optimization, which typically applies only to smooth equality constraints. Evaluated on large-scale dynamic planning tasks, the proposed approach successfully generates dynamically feasible trajectories and demonstrates superior robustness and solvability in scenarios where standard algorithms fail.