🤖 AI Summary
This work addresses the challenge of Bayesian optimization in mixed search spaces containing both continuous and non-uniformly spaced discrete variables, which commonly arise in natural sciences. Existing methods are hindered by the unavailability of gradients and the difficulty of optimizing acquisition functions over such domains. The authors propose the first extension of probabilistic reparameterization to non-equidistant discrete variables, enabling gradient-based optimization across the entire mixed space by integrating Gaussian process surrogates with tailored kernel functions. The method demonstrates robust performance on highly discontinuous and discretized objective functions, significantly improving sample efficiency over synthetic benchmarks and real-world scientific experiments. It is particularly well-suited for data-scarce, high-noise settings such as autonomous laboratories.
📝 Abstract
Optimizing expensive black-box objectives over mixed search spaces is a common challenge across the natural sciences. Bayesian optimization (BO) offers sample-efficient strategies through probabilistic surrogate models and acquisition functions. However, its effectiveness diminishes in mixed or high-cardinality discrete spaces, where gradients are unavailable and optimizing the acquisition function becomes computationally demanding. In this work, we generalize the probabilistic reparameterization (PR) approach of Daulton et al. to handle non-equidistant discrete variables, enabling gradient-based optimization in fully mixed-variable settings with Gaussian process (GP) surrogates. With real-world scientific optimization tasks in mind, we conduct systematic benchmarks on synthetic and experimental objectives to obtain an optimized kernel formulations and demonstrate the robustness of our generalized PR method. We additionally show that, when combined with a modified BO workflow, our approach can efficiently optimize highly discontinuous and discretized objective landscapes. This work establishes a practical BO framework for addressing fully mixed optimization problems in the natural sciences, and is particularly well suited to autonomous laboratory settings where noise, discretization, and limited data are inherent.