Nonsmooth Nonconvex-Nonconcave Minimax Optimization: Primal-Dual Balancing and Iteration Complexity Analysis

2022-09-22

AI Summary

This work addresses nonsmooth, nonconvex–nonconcave minimax optimization—problems where standard methods fail due to their reliance on smoothness and concavity assumptions. Method: We propose the Smoothed Proximal Linear Descent-Ascent (Smoothed PLDA) algorithm, the first to handle such nonsmooth settings without smoothing the objective or imposing structural convexity/concavity. Contribution/Results: We introduce a novel nonsmooth primal-dual error bound analysis framework and establish a unified iteration complexity of (O(varepsilon^{-2max{2 heta,1}})) under the Kurdyka–Łojasiewicz (KL) condition with exponent ( heta in [0,1)); this achieves the optimal rate (O(varepsilon^{-2})) when ( heta in [0,1/2]). Theoretically, Smoothed PLDA converges at the optimal rate to both (varepsilon)-game stationary points and (varepsilon)-optimization stationary points. Moreover, we quantitatively characterize equivalences among multiple stationarity notions for the first time. We further verify that max-structured problems satisfy the KL condition with ( heta = 0), thereby establishing a new analytical paradigm and an efficient solver for nonsmooth minimax optimization.

Abstract

Nonconvex-nonconcave minimax optimization has gained widespread interest over the last decade. However, most existing works focus on variants of gradient descent-ascent (GDA) algorithms, which are only applicable to smooth nonconvex-concave settings. To address this limitation, we propose a novel algorithm named smoothed proximal linear descent-ascent (smoothed PLDA), which can effectively handle a broad range of structured nonsmooth nonconvex-nonconcave minimax problems. Specifically, we consider the setting where the primal function has a nonsmooth composite structure and the dual function possesses the Kurdyka-Lojasiewicz (KL) property with exponent $ heta in [0,1)$. We introduce a novel convergence analysis framework for smoothed PLDA, the key components of which are our newly developed nonsmooth primal error bound and dual error bound. Using this framework, we show that smoothed PLDA can find both $epsilon$-game-stationary points and $epsilon$-optimization-stationary points of the problems of interest in $mathcal{O}(epsilon^{-2max{2 heta,1}})$ iterations. Furthermore, when $ heta in [0,frac{1}{2}]$, smoothed PLDA achieves the optimal iteration complexity of $mathcal{O}(epsilon^{-2})$. To further demonstrate the effectiveness and wide applicability of our analysis framework, we show that certain max-structured problem possesses the KL property with exponent $ heta=0$ under mild assumptions. As a by-product, we establish algorithm-independent quantitative relationships among various stationarity concepts, which may be of independent interest.

Problem

• Complex Optimization
• Nonconvex-Nonconcave Minimax Optimization
• Efficiency Analysis

Innovation

• Smoothed Proximal Linear Descent-Ascent
• Nonconvex-Nonconcave Minimax Optimization
• Analysis Framework