🤖 AI Summary
This study addresses the miscalibration of Gaussian processes (GPs) in uncertainty quantification (UQ)—specifically, their frequent lack of probabilistic calibration, which undermines convergence in downstream tasks such as Bayesian optimization. To tackle this, we propose Kernel Covariance Validation (KCV), the first framework to systematically exploit the multivariate normal structure of GP predictions for interpretable, computationally tractable calibration diagnostics. KCV integrates multivariate normality testing, uncertainty calibration analysis, and adaptive target design to quantitatively detect and localize model misspecification. Extensive experiments across 1D to high-dimensional GPs demonstrate KCV’s ability to identify canonical calibration failure modes. Results show that KCV significantly improves predictive reliability and enhances both the convergence speed and stability of optimization algorithms. By enabling principled, diagnostic-driven calibration, KCV establishes a new paradigm for trustworthy UQ in GP-based modeling.
📝 Abstract
Gaussian Process (GP) models are popular tools in uncertainty quantification (UQ) because they purport to furnish functional uncertainty estimates that can be used to represent model uncertainty. It is often difficult to state with precision what probabilistic interpretation attaches to such an uncertainty, and in what way is it calibrated. Without such a calibration statement, the value of such uncertainty estimates is quite limited and qualitative. We motivate the importance of proper probabilistic calibration of GP predictions by describing how GP predictive calibration failures can cause degraded convergence properties in a target optimization algorithm called Targeted Adaptive Design (TAD). We discuss the interpretation of GP-generated uncertainty intervals in UQ, and how one may learn to trust them, through a formal procedure for covariance kernel validation that exploits the multivariate normal nature of GP predictions. We give simple examples of GP regression misspecified 1-dimensional models, and discuss the situation with respect to higher-dimensional models.