Bayesian Optimization on Function Spaces via Sparse RKHS Manifolds

2026-10-05

AI Summary

This study addresses the limitation of traditional Bayesian optimization to finite-dimensional vectors, which hinders black-box optimization in infinite-dimensional function spaces. To overcome this, we propose a functional Bayesian optimization method based on L0 manifold optimization. By jointly optimizing kernel locations and coefficients within sparse subspaces of reproducing kernel Hilbert spaces (RKHS), our approach effectively circumvents the dimensionality bottleneck. Key contributions include establishing a unified theoretical perspective that elucidates existing methods and developing a novel test function suite that extends finite-dimensional benchmarks to infinite-dimensional domains. Extensive experiments demonstrate that the proposed method achieves superior overall performance compared to state-of-the-art techniques across a wide range of benchmarks, thereby validating its effectiveness for optimization in infinite-dimensional spaces.

Abstract

Bayesian Optimization (BO) has become an established methodology for minimizing black-box functions of a vector input. Often, however, this parameter vector arises from the discretization of an inherently functional relationship. Several recent articles have considered the Functional Bayesian Optimization (FBO) setting, in which the variable to be optimized is not a member of a finite dimensional vector space, but rather an infinite dimensional function space. In this work, we propose $L^0$ Manifold Optimization (L0MO), a simple approach to FBO which searches the subset of a Reproducing Kernel Hilbert Space (RKHS) consisting of functions with a sparse representation in the kernel functions, optimizing both the kernel locations and their coefficients. We discuss in detail the relationship between our method and existing ones, providing a unifying lens through which to view prior works. To assess our method against the state of the art, we conduct an extensive computational study, and along the way develop a novel set of benchmark test functions which port standard finite-dimensional ones to the infinite dimensional domain. Our experiments demonstrate that, on balance, the proposed method achieves superior performance across a wide range of test benchmarks.

Problem

• Bayesian Optimization
• Function Spaces
• Functional Bayesian Optimization
• Infinite-dimensional Optimization
• Black-box Optimization

Innovation

• Bayesian Optimization
• Function Spaces
• Sparse RKHS
• Manifold Optimization
• Benchmark Functions