On Thompson Sampling and Bilateral Uncertainty in Additive Bayesian Optimization

2025-10-13

AI Summary

In Bayesian optimization (BO), additive models alleviate the curse of dimensionality in modeling and search, but standard acquisition functions—e.g., additive upper confidence bound (UCB)—ignore pairwise interdimensional covariances (“bilateral uncertainty”, BU), introducing non-asymptotic approximation error. Method: We investigate the practical impact of this approximation under finite evaluation budgets and propose an efficient Thompson sampling scheme grounded in conditional independence decomposition, the first to rigorously preserve BU within additive Gaussian processes. Contribution/Results: Empirical evaluation across low-budget, high-dimensional benchmarks shows that conventional BU-ignoring methods perform statistically indistinguishable from exact BU-aware counterparts. These results validate the practical sufficiency of existing simplifications in non-asymptotic regimes, providing critical empirical support for both the theoretical soundness and engineering viability of additive BO.

Technology Category

Reasoning under Uncertainty: Stochastic OptimizationSearch and Optimization: Sampling/Simulation-based SearchMachine Learning: Calibration & Uncertainty Quantification

Application Category

User Modeling, Personalization and Recommendation: Fairness-aware retrieval and rankingGraph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In Bayesian Optimization (BO), additive assumptions can mitigate the twin difficulties of modeling and searching a complex function in high dimension. However, common acquisition functions, like the Additive Lower Confidence Bound, ignore pairwise covariances between dimensions, which we'll call extit{bilateral uncertainty} (BU), imposing a second layer of approximations. While theoretical results indicate that asymptotically not much is lost in doing so, little is known about the practical effects of this assumption in small budgets. In this article, we show that by exploiting conditional independence, Thompson Sampling respecting BU can be efficiently conducted. We use this fact to execute an empirical investigation into the loss incurred by ignoring BU, finding that the additive approximation to Thompson Sampling does indeed have, on balance, worse performance than the exact method, but that this difference is of little practical significance. This buttresses the theoretical understanding and suggests that the BU-ignoring approximation is sufficient for BO in practice, even in the non-asymptotic regime.

Problem

• Analyzing bilateral uncertainty effects in additive Bayesian Optimization
• Evaluating Thompson Sampling performance with additive approximations
• Assessing practical significance of ignoring pairwise covariances

Innovation

• Thompson Sampling with bilateral uncertainty
• Exploiting conditional independence for efficiency
• Empirical comparison of additive approximation performance