Every Call is Precious: Global Optimization of Black-Box Functions with Unknown Lipschitz Constants

2025-02-06

AI Summary

We address global optimization of expensive, non-convex, black-box functions that are Lipschitz continuous with an unknown constant. We propose ECP, an efficient algorithm requiring no prior knowledge or estimation of the Lipschitz constant. ECP employs adaptive domain contraction guided by confidence upper bounds, integrating worst-case analysis with dynamic scale-aware exploration. Theoretically, we establish the first no-regret global optimization framework under finite evaluation budgets, achieving a tighter optimal regret bound. Empirically, ECP outperforms ten state-of-the-art methods—including Bayesian optimization, multi-armed bandits, and evolutionary algorithms—across 30 synthetic and real-world multidimensional non-convex benchmarks, demonstrating superior robustness and SOTA performance.

Technology Category

• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Optimizing expensive, non-convex, black-box Lipschitz continuous functions presents significant challenges, particularly when the Lipschitz constant of the underlying function is unknown. Such problems often demand numerous function evaluations to approximate the global optimum, which can be prohibitive in terms of time, energy, or resources. In this work, we introduce Every Call is Precious (ECP), a novel global optimization algorithm that minimizes unpromising evaluations by strategically focusing on potentially optimal regions. Unlike previous approaches, ECP eliminates the need to estimate the Lipschitz constant, thereby avoiding additional function evaluations. ECP guarantees no-regret performance for infinite evaluation budgets and achieves minimax-optimal regret bounds within finite budgets. Extensive ablation studies validate the algorithm's robustness, while empirical evaluations show that ECP outperforms 10 benchmark algorithms including Lipschitz, Bayesian, bandits, and evolutionary methods across 30 multi-dimensional non-convex synthetic and real-world optimization problems, which positions ECP as a competitive approach for global optimization.

Problem

• Optimize black-box functions
• Unknown Lipschitz constants
• Minimize function evaluations

Innovation

• Global optimization without Lipschitz constant
• Minimizes unpromising function evaluations
• Outperforms 10 benchmark algorithms