The First-Order Oracle Complexity of Lipschitz Convex Optimization in Nondual Settings

2026-09-17

AI Summary

研究了在非对偶设置下，通过新的在线学习方法解决了Lipschitz凸优化问题，特别是对于较小的可行集（p<q），证明了几何结构能够改善收敛速度。

Abstract

We study first-order black-box convex optimization over an $\ell_p$-ball for objectives Lipschitz in the $\ell_q$-norm, solving in the affirmative the nonsmooth version of the COLT open question (Guz15b) on whether the geometry of a smaller feasible set ($p < q$) can improve convergence rates in convex optimization, and matching prior lower bounds up to logarithmic factors. Our rates include \(\widetilde O(1/T)\) for convex Euclidean-Lipschitz optimization over the $\ell_1$-ball, improving on the $O(1/\sqrt{T})$ classical rate under general assumptions. The key technical device is a new online learning game, where the comparator is evaluated using the maximum of affine losses observed so far. We bound the value of this game above and below in terms of a combinatorial online learning quantity: the sequential fat-shattering dimension, which we characterize for the $\ell_p / \ell_q$ case. Our results generally apply when the feasible set $X$ and the set of possible subgradients $H$ are convex, centrally symmetric, and admit a type of minmax theorem, advancing on a fundamental question by Sridharan [Sri12, Section 10.1.2, Q3]. As a geometric consequence of our analysis, of independent interest, we obtain estimates for the expected distance of a convex hull of samples to their mean in several Banach geometries, a version of the celebrated Wendel's theorem (Wen62), but quantitative and for bounded general distributions as opposed to centrally symmetric ones.

Problem

• First-Order Oracle Complexity
• Lipschitz Convex Optimization
• Nondual Settings

Innovation

• First-Order Oracle Complexity
• Online Learning Game
• Sequential Fat-Shattering Dimension
• Convex Optimization over ell_p-ball
• Wendel's Theorem