Sharp First-Order Lower Bounds for Higher-Order Smooth Nonconvex Optimization

2026-06-03

AI Summary

This work investigates the optimal query complexity lower bounds for first-order gradient oracles in finding ε-stationary points of higher-order smooth nonconvex optimization problems. By introducing a novel “block-chain” mechanism to construct hard instances and integrating higher-order smoothness analysis with deterministic first-order complexity theory, the study establishes the first dimension-free tight lower bounds: Ω(ε⁻⁷/⁴) under Hessian-Lipschitz continuity and Ω(ε⁻⁵/³) in the third-order smooth setting. These bounds match the known upper bounds achieved by existing accelerated algorithms, thereby resolving a long-standing gap in the theoretical understanding of nonconvex optimization complexity.

Abstract

We study the deterministic first-order oracle complexity of finding \(ε\)-stationary points in smooth nonconvex optimization when the objective satisfies higher-order smoothness assumptions. While the classical \(ε^{-2}\) rate is optimal under only Lipschitz gradients, higher-order smoothness leads to accelerated first-order upper bounds, most notably the \(ε^{-7/4}\) rate under Lipschitz Hessians and the \(ε^{-5/3}\) rate under Lipschitz third derivatives. The matching lower bounds, however, have remained open. We resolve this gap by proving a new dimension-free first-order lower bound for higher-order smooth nonconvex functions, valid for every finite smoothness order. In particular, our construction gives a matching \(Ω(ε^{-7/4})\) lower bound in the Hessian-Lipschitz case and a matching \(Ω(ε^{-5/3})\) lower bound in the third-order-smooth regime. The hard instance is based on a \emph{block-chain} mechanism that enforces blockwise oracle revelation while preserving the smoothness structure needed for the scalar hard instance. The lower-bound construction was discovered with the assistance of ChatGPT 5.5 Pro and subsequently verified by the authors.

Problem

• nonconvex optimization
• first-order oracle complexity
• higher-order smoothness
• lower bounds
• ε-stationary points

Innovation

• first-order lower bounds
• higher-order smoothness
• nonconvex optimization
• block-chain construction
• oracle complexity