Wall-Clock Complexity for Zeroth-Order Optimization with Tunable Oracle Fidelity

2026-05-29

AI Summary

This work addresses zeroth-order optimization where the objective function is evaluated via computationally expensive simulations with tunable accuracy. It introduces a wall-clock time model that jointly accounts for query precision and computational cost, enabling a systematic analysis of various noise models, oracle types, and optimization algorithms. The study derives optimal strategies for accuracy scheduling and batch sizing, revealing—perhaps counterintuitively—that accelerated methods can underperform their non-accelerated counterparts in actual runtime. It further establishes sufficient conditions under which constant-precision strategies are asymptotically optimal in the big-O sense. The proposed unified framework automatically recommends optimal parameters based on algorithmic and noise characteristics, significantly reducing total computation time while preserving solution accuracy.

Abstract

Zeroth-order (black-box) optimization is applied when gradients are unavailable and objective evaluations rely on expensive simulations. In many such applications, the oracle fidelity is tunable: higher-accuracy queries reduce noise but incur higher computational costs. To capture this trade-off, we study an accuracy-aware wall-clock model where each query with fidelity $δ$ has a cost $c(δ)$, and we minimize the total time $T_{\mathrm{total}} = \sum_{k=1}^{N} c(δ_k)$, subject to a target accuracy constraint. We show how the choice of oracle type, noise model, and optimization scheme induces explicit wall-clock-optimal choices for the algorithmic parameters. For instance, we demonstrate that accelerated methods can be wall-clock inferior to non-accelerated schemes. Furthermore, we characterize the conditions under which a constant fidelity strategy is optimal in the Big-O sense. Our framework provides a unified methodology to translate convergence guarantees into practical fidelity and batching recommendations.

Problem

• zeroth-order optimization
• oracle fidelity
• wall-clock complexity
• accuracy-cost trade-off
• black-box optimization

Innovation

• zeroth-order optimization
• tunable oracle fidelity
• wall-clock complexity
• accuracy-aware optimization
• query cost trade-off