🤖 AI Summary
This paper studies optimization problems with weakly convex objective functions subject to general convex (linear or nonlinear) constraints. We propose the Damped Proximal Augmented Lagrangian Method (DPALM), the first augmented Lagrangian framework incorporating a damping mechanism—specifically, damped dual step sizes—to ensure boundedness of dual variables and naturally accommodate nonlinear convex constraints. Theoretically, under weak convexity, DPALM achieves a tight outer-loop complexity of $O(varepsilon^{-2})$ and an overall complexity of $widetilde{O}(varepsilon^{-2.5})$ to attain a KKT residual $leq varepsilon$, improving upon the best-known bounds. Further integrating Moreau envelope smoothing with accelerated proximal gradient (APG) yields an overall complexity of $widetilde{O}(varepsilon^{-3})$. Experiments demonstrate that DPALM significantly outperforms state-of-the-art methods on nonconvex quadratic programming and robust nonlinear least-squares tasks.
📝 Abstract
We give a damped proximal augmented Lagrangian method (DPALM) for solving problems with a weakly-convex objective and convex linear/nonlinear constraints. Instead of taking a full stepsize, DPALM adopts a damped dual stepsize to ensure the boundedness of dual iterates. We show that DPALM can produce a (near) $vareps$-KKT point within $O(vareps^{-2})$ outer iterations if each DPALM subproblem is solved to a proper accuracy. In addition, we establish overall iteration complexity of DPALM when the objective is either a regularized smooth function or in a regularized compositional form. For the former case, DPALM achieves the complexity of $widetilde{mathcal{O}}left(varepsilon^{-2.5}
ight)$ to produce an $varepsilon$-KKT point by applying an accelerated proximal gradient (APG) method to each DPALM subproblem. For the latter case, the complexity of DPALM is $widetilde{mathcal{O}}left(varepsilon^{-3}
ight)$ to produce a near $varepsilon$-KKT point by using an APG to solve a Moreau-envelope smoothed version of each subproblem. Our outer iteration complexity and the overall complexity either generalize existing best ones from unconstrained or linear-constrained problems to convex-constrained ones, or improve over the best-known results on solving the same-structured problems. Furthermore, numerical experiments on linearly/quadratically constrained non-convex quadratic programs and linear-constrained robust nonlinear least squares are conducted to demonstrate the empirical efficiency of the proposed DPALM over several state-of-the art methods.