Frugal Bayesian Optimization: Scalable Surrogates for Data- and Resource-Limited Discovery

2026-07-31

AI Summary

This work addresses the longstanding challenge in Bayesian optimization of balancing performance and computational efficiency, which is often overlooked due to the neglect of computational costs. The study introduces the first reproducible benchmark that jointly evaluates optimization quality and computational overhead, systematically assessing surrogate models including Gaussian processes, random forests, NGBoost, and Bayesian adaptive splines. Furthermore, it proposes a lightweight, data-characteristic-driven strategy for surrogate model selection. Extensive experiments across eight synthetic benchmark functions and nine real-world datasets demonstrate that non-Gaussian process surrogates can achieve comparable or even superior optimization performance while substantially reducing computational cost, thereby breaking the reliance on expensive Gaussian process models.

Abstract

Bayesian Optimization (BO) is widely adopted for data-efficient optimization in scientific and engineering applications, yet its computational cost is rarely evaluated alongside optimization performance. Here we present a systematic, compute-aware study of BO that evaluates surrogate models along two axes: optimization quality and computational frugality. Across eight benchmark functions and nine real-world datasets spanning materials science, mechanics, robotics, chemistry, and machine learning, we benchmark four surrogate models: Gaussian Processes, Random Forests, NGBoost, and Bayesian Adaptive Spline Surfaces. We show that Gaussian Process-based BO consistently incurs the highest time and memory overhead without delivering superior optimization or sample efficiency. In contrast, scalable alternatives achieve equal or better performance at a fraction of the computational cost. Motivated by these findings, we introduce a surrogate-recommendation framework that predicts the most suitable BO surrogate from inexpensive dataset characteristics. Together, these results establish FruBO as a reproducible, compute-aware baseline for Bayesian Optimization and provide practical guidance for surrogate selection under limited computational and experimental budgets.

Problem

• Bayesian Optimization
• computational frugality
• surrogate models
• resource-limited discovery
• data-efficient optimization

Innovation

• Frugal Bayesian Optimization
• Surrogate Models
• Compute-aware Optimization
• Scalable BO
• Surrogate Recommendation