Near-Optimal Convex Optimization with Lazy Second-Order Oracles

2026-10-02

AI Summary

This study addresses the iteration complexity of convex optimization under a lazy second-order oracle. Methodologically, it employs a blockwise zero-chain construction technique to rigorously establish complexity lower bounds and proposes a query strategy that yields a nearly optimal matching algorithm. The primary contribution lies in significantly improving existing upper bound results and establishing tight complexity bounds up to logarithmic factors. By doing so, this work provides a near-complete characterization of the computational complexity for this optimization model.

Abstract

This paper studies the complexity of convex optimization using lazy second-order oracles (Doikov, Chayti, and Jaggi, ICML 2023), where an algorithm queries gradients every iteration and Hessians once per $m$ iterations. Under this setting, we show a lower bound of $Ω(m+ m^{1/7} ε^{-2/7})$ on the number of total iterations to find an $ε$-solution using a novel block zero-chain construction. Then we propose a novel method that achieves a new upper bound of $\tilde{\mathcal{O}}(m+ m^{1/7} ε^{-2/7})$, which significantly improves the prior one (Chen, Liu, Luo, and Zhang, COLT 2026) of $\tilde{\mathcal{O}}(m+ m^{13/21} ε^{-2/7})$ and is tight up to logarithmic factors.

Problem

• convex optimization
• lazy second-order oracles
• iteration complexity
• lower bound

Innovation

• Lazy Second-Order Oracles
• Convex Optimization
• Block Zero-Chain Construction
• Iteration Complexity
• Near-Optimal Upper Bound