🤖 AI Summary
For expensive black-box function optimization—e.g., hyperparameter tuning of large language models—this paper proposes Bayesian Distance Correlation (BDC), a novel Bayesian optimization framework grounded in distance correlation. BDC innovatively incorporates distance correlation into the acquisition function design, enabling automatic, hyperparameter-free balancing of exploration and exploitation without relying on prior assumptions or manual tuning. It integrates Gaussian process regression with sequential integral observations modeling. Empirical evaluation across multiple benchmark tasks shows BDC matches the performance of Expected Improvement (EI) and Max-value Entropy Search (MES); it further demonstrates superior efficiency and robustness in sequential observation tasks over unknown landscapes. The core contribution lies in replacing conventional heuristic acquisition criteria with a data-driven, interpretable distance correlation measure—establishing a new, parameter-free paradigm for expensive function optimization.
📝 Abstract
The need to collect data via expensive measurements of black-box functions is prevalent across science, engineering and medicine. As an example, hyperparameter tuning of a large AI model is critical to its predictive performance but is generally time-consuming and unwieldy. Bayesian optimization (BO) is a collection of methods that aim to address this issue by means of Bayesian statistical inference. In this work, we put forward a BO scheme named BDC, which integrates BO with a statistical measure of association of two random variables called Distance Correlation. BDC balances exploration and exploitation automatically, and requires no manual hyperparameter tuning. We evaluate BDC on a range of benchmark tests and observe that it performs on per with popular BO methods such as the expected improvement and max-value entropy search. We also apply BDC to optimization of sequential integral observations of an unknown terrain and confirm its utility.