Problem
Research questions and friction points this paper is trying to address.
Preferential Bayesian Optimization
high-dimensional optimization
pairwise preference feedback
cumulative regret
policy search
Local Preferential Bayesian Optimization
🤖 AI Summary
This work addresses the challenges of high-dimensional Bayesian optimization with preference feedback, where global search strategies often suffer from low efficiency and struggle to locate complex, sharp optima. To overcome these limitations, the authors propose Local Preference Bayesian Optimization (LPBO), which introduces a local search mechanism into the preference learning framework for the first time. By constructing a trust region and leveraging derivative information from the Gaussian process posterior—obtained via Laplace approximation to estimate both first- and second-order derivatives—the method enables efficient and accurate high-dimensional optimization. Empirical evaluations demonstrate that LPBO significantly reduces cumulative regret compared to existing global preference-based baselines across synthetic GP sample paths, standard benchmark functions, and policy search tasks.
📝 Abstract
Bayesian optimization (BO) is a popular and effective approach for tuning expensive, noisy experiments, but requires the formulation of an explicit objective function. Preferential BO (PBO) removes this requirement by learning from pairwise human feedback, yet existing methods struggle to efficiently optimize beyond low- and medium-dimensional problems due to their global search approaches. We address this limitation by developing a family of local PBO methods that transfer key ideas from high-dimensional BO to the preferential setting. In particular, we introduce local PBO methods which adapt trust-region and derivative-informed local search to pairwise preference feedback, where the latter exploits first- and second-order derivatives of the Laplace-approximated GP posterior. Our benchmark on GP sample paths, standard optimization benchmark functions, and policy-search tasks shows that local PBO methods are especially effective in high-dimensional and complex landscapes with steep optima. Compared with global preference-based baselines, they can substantially reduce cumulative regret, making them particularly useful for real-world preference-based optimization tasks such as policy search.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Preferential Bayesian Optimization
Local Search
Trust-Region
Derivative-Informed Optimization
High-Dimensional Optimization
🔎 Similar Papers
No similar papers found.
