🤖 AI Summary
This work addresses the limitations of traditional alternating direction method of multipliers (ADMM) and block coordinate descent (BCD) methods in large-scale optimization, particularly regarding computational efficiency and convergence guarantees. The authors propose an adaptive proximal ADMM algorithm along with two BCD variants that accommodate inexact subproblem solutions. A key innovation is the introduction of an inexact proximal mapping with dynamic error control, and the paper establishes, for the first time, convergence rate guarantees for stochastic BCD under Hölder smoothness assumptions. Theoretical analysis demonstrates that the proposed algorithms achieve optimal iteration complexity, matching the best-known rates for Lipschitz-smooth settings across nonconvex, convex, and strongly convex cases. Numerical experiments further confirm that the dynamic error strategy significantly outperforms fixed-error approaches.
📝 Abstract
This dissertation explores block decomposable methods for large-scale optimization problems. It focuses on alternating direction method of multipliers (ADMM) schemes and block coordinate descent (BCD) methods. Specifically, it introduces a new proximal ADMM algorithm and proposes two BCD methods. The first part of the research presents a new proximal ADMM algorithm. This method is adaptive to all problem parameters and solves the proximal augmented Lagrangian (AL) subproblem inexactly. This adaptiveness facilitates the highly efficient application of the algorithm to a broad swath of practical problems. The inexact solution of the proximal AL subproblem overcomes many key challenges in the practical applications of ADMM. The resultant algorithm obtains an approximate solution of an optimization problem in a number of iterations that matches the state-of-the-art complexity for the class of proximal ADMM schemes. The second part of the research focuses on an inexact proximal mapping for the class of block proximal gradient methods. Key properties of this operator is established, facilitating the derivation of convergence rates for the proposed algorithm. Under two error decreases conditions, the algorithm matches the convergence rate of its exactly computed counterpart. Numerical results demonstrate the superior performance of the algorithm under a dynamic error regime over a fixed one. The dissertation concludes by providing convergence guarantees for the randomized BCD method applied to a broad class of functions, known as H\"older smooth functions. Convergence rates are derived for non-convex, convex, and strongly convex functions. These convergence rates match those furnished in the existing literature for the Lipschtiz smooth setting.