🤖 AI Summary
This work studies the cumulative regret of one-dimensional noisy Bayesian optimization (BO) under Gaussian process priors and Gaussian observation noise. Methodologically, it employs information-theoretic analysis, RKHS complexity estimation, and confidence interval construction. The key contribution is the first nontrivial lower bound on regret, tightly complementing the classical upper bound of Srinivas et al. (2009): for the squared-exponential (SE) kernel, the regret is shown to be Ω(√T) and O(√(T log T)), achieving near-tight characterization (up to a √log T factor); for the Matérn-ν kernel with ν > 2, the existing upper bound is proven strictly suboptimal and improvable. These results provide the tightest known characterization of regret growth for one-dimensional noisy BO and reveal the fundamental role of kernel smoothness in determining regret bounds.
📝 Abstract
We consider the problem of Bayesian optimization (BO) in one dimension, under a Gaussian process prior and Gaussian sampling noise. We provide a theoretical analysis showing that, under fairly mild technical assumptions on the kernel, the best possible cumulative regret up to time $T$ behaves as $Omega(sqrt{T})$ and $O(sqrt{Tlog T})$. This gives a tight characterization up to a $sqrt{log T}$ factor, and includes the first non-trivial lower bound for noisy BO. Our assumptions are satisfied, for example, by the squared exponential and Mat'ern-$
u$ kernels, with the latter requiring $
u > 2$. Our results certify the near-optimality of existing bounds (Srinivas {em et al.}, 2009) for the SE kernel, while proving them to be strictly suboptimal for the Mat'ern kernel with $
u > 2$.