🤖 AI Summary
This work addresses the sensitivity of Bayesian optimization to model misspecification, which can lead to fragile out-of-sample decisions. The authors propose a distributionally robust optimization framework that formalizes and quantifies model robustness under perturbations to both parameters and likelihood through novel measures: posterior sensitivity and likelihood sensitivity. Theoretical analysis reveals that posterior sensitivity vanishes as variance decreases, whereas likelihood sensitivity persists; parameter learning mitigates the former but cannot eliminate the latter. By constructing an uncertainty set based on a bias-aware divergence measure, the method achieves a near-Pareto-optimal trade-off between expected performance and dual robustness. Empirical experiments validate the effectiveness of the proposed approach.
📝 Abstract
We introduce the notion of worst-case posterior and worst-case likelihood sensitivity. These measure, respectively, the sensitivity of the expected cost to worst-case perturbations of the posterior distribution and worst-case perturbations of the likelihood of a Bayesian model. Each defines a quantitative measure of robustness. A decision maker concerned about the sensitivity of the out-of-sample expected cost to deviations from her assumptions will want a decision for which both sensitivities are small. We derive posterior and likelihood sensitivities for uncertainty sets defined in terms of deviation measures. Posterior sensitivity vanishes when the posterior variance shrinks to zero, which occurs when parameter uncertainty is eliminated from learning. Parameter learning does not eliminate likelihood sensitivity. A distributionally robust formulation of a Bayesian optimization problem makes a near-Pareto-optimal tradeoff between performance (expected cost) and robustness (posterior and likelihood sensitivity).