Out-Of-The-Loop Multi-Fidelity Bayesian Optimization

2026-08-04

AI Summary

This work addresses a practical challenge in multi-fidelity Bayesian optimization where the highest-fidelity function is too computationally expensive to evaluate within the optimization loop, yet historical high-fidelity observations are available. The authors propose an improved approach that effectively incorporates these historical high-fidelity data by introducing task descriptors—either explicitly provided or extracted from unstructured metadata. This method reveals, for the first time, the suboptimality of standard multi-fidelity Bayesian optimization in such settings and innovatively enables cross-task information transfer. Empirical evaluations on synthetic benchmarks, molecular design, and hyperparameter tuning demonstrate that the proposed framework consistently outperforms existing methods, offering both strong effectiveness and practical utility.

Abstract

Black-box optimization is a ubiquitous problem in science and engineering, often dealing with expensive objective functions with cheaper lower-fidelity proxies available. Multi-fidelity Bayesian optimization (MF-BO) is a principled approach to this problem, leveraging correlations across different fidelities when querying the objective. However, for many important MF-BO tasks, the true highest-fidelity function is prohibitively expensive to be part of the optimization loop. Nevertheless, practitioners often have gold standard data (observations of the highest-fidelity function) obtained from previous experiments that might provide information for the current task. For instance, in molecular optimization, chemists often pick the top-$k$ candidate molecules using various computer simulations, and later reveal their true objective function values. In this work, we demonstrate the suboptimality of standard MF-BO algorithms in the real-world scenarios above, even under ideal assumptions. Next, we mitigate this problem by incorporating historical high-fidelity data accompanied by task descriptors---which can be explicitly given or extracted from unstructured metadata. We demonstrate the effectiveness of our methods on synthetic functions, as well as real-world problems in chemistry and hyperparameter optimization.

Problem

• Multi-fidelity Bayesian Optimization
• Out-Of-The-Loop
• Black-box Optimization
• High-fidelity Data
• Expensive Objective Functions

Innovation

• Multi-fidelity Bayesian Optimization
• Out-Of-The-Loop
• Historical High-Fidelity Data
• Task Descriptors
• Black-box Optimization