🤖 AI Summary
This work addresses the issue of miscalibration in probabilistic regression, where predictive distributions often exhibit overconfidence—particularly problematic in safety-critical applications—due to an emphasis on informativeness at the expense of calibration. The authors propose a nonparametric post-hoc calibration method based on conditional kernel mean embeddings, which effectively corrects calibration bias without requiring parametric assumptions about the error structure. By introducing a novel characteristic kernel tailored for real-valued targets, the approach overcomes limitations of existing techniques that rely either on weak calibration criteria or strong parametric assumptions. The method achieves efficient empirical distribution calibration with a computational complexity of $\mathcal{O}(n \log n)$. Extensive experiments across diverse regression benchmarks and models demonstrate that the proposed technique significantly outperforms current post-hoc calibration methods, substantially enhancing the reliability of predictive distributions.
📝 Abstract
A key challenge in probabilistic regression is ensuring that predictive distributions accurately reflect true empirical uncertainty. Minimizing overall prediction error often encourages models to prioritize informativeness over calibration, producing narrow but overconfident predictions. However, in safety-critical settings, trustworthy uncertainty estimates are often more valuable than narrow intervals. Realizing the problem, several recent works have focused on post-hoc corrections; however, existing methods either rely on weak notions of calibration (such as PIT uniformity) or impose restrictive parametric assumptions on the nature of the error. To address these limitations, we propose a novel nonparametric re-calibration algorithm based on conditional kernel mean embeddings, capable of correcting calibration error without restrictive modeling assumptions. For efficient inference with real-valued targets, we introduce a novel characteristic kernel over distributions that can be evaluated in $\mathcal{O}(n \log n)$ time for empirical distributions of size $n$. We demonstrate that our method consistently outperforms prior re-calibration approaches across a diverse set of regression benchmarks and model classes.