Problem
Research questions and friction points this paper is trying to address.
Bayesian Optimization
Hyperparameter Tuning
Cumulative Regret
Black-box Optimization
Pre-training of Bayesian Optimization Algorithm through Bayesian Optimization
🤖 AI Summary
This study addresses the challenge that the performance of Bayesian optimization (BO) heavily depends on hyperparameter presets by proposing a data-driven bilevel BO framework for automatic parameter tuning. Methodologically, it introduces a novel pretraining paradigm that infers Gaussian processes from initial observations and generates sample paths. The outer level employs cumulative regret as the evaluation metric to automatically search for optimal hyperparameter configurations via Bayesian optimization. Experimental results demonstrate that this bilevel architecture efficiently identifies highly robust hyperparameter combinations from the candidate space, significantly reducing the cost of manual tuning while enhancing overall optimization efficiency.
📝 Abstract
Bayesian optimization (BO) is widely used as a standard approach for expensive black-box optimization. However, BO algorithms often involve parameters that must be specified in advance, and their performance can strongly depend on these choices. We propose a framework for optimizing such parameters using sample paths drawn from a Gaussian process (GP) inferred from the information available at the start of BO. We use cumulative regret as the performance metric for a BO algorithm. By running the BO algorithm on the generated sample paths, we obtain an empirical estimate of its expected cumulative regret for a given parameter configuration. Optimizing this estimate allows us to identify parameter configurations that, given the currently available information, are expected to achieve low cumulative regret. Since this parameter optimization is itself a black-box optimization problem, we employ another BO procedure to solve it, which we refer to as outer BO. Through experiments, we demonstrate that the proposed framework can effectively select parameter configurations that achieve strong performance among a range of candidate configurations.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Bayesian Optimization
Hyperparameter Tuning
Gaussian Process
Cumulative Regret
Meta-optimization
🔎 Similar Papers
No similar papers found.
