Uncertainty Quantification for Regression: A Unified Framework based on kernel scores

2025-10-29

AI Summary

Reliable uncertainty quantification remains critical—especially in safety-critical regression tasks—yet existing methods predominantly target classification, leaving regression uncertainty modeling underdeveloped. This paper introduces a unified uncertainty quantification framework grounded in kernel scores, the first to systematically support joint estimation of total, aleatoric, and epistemic uncertainties. By explicitly modeling kernel function properties, we establish analytical connections between kernel characteristics and key desiderata: tail sensitivity, robustness, and out-of-distribution (OOD) detection capability—yielding interpretable, designable uncertainty measures. Experiments demonstrate superior downstream performance: different kernel choices enable explicit, controllable trade-offs in uncertainty estimation, and our approach significantly outperforms state-of-the-art methods in robust regression and OOD detection.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Regression tasks, notably in safety-critical domains, require proper uncertainty quantification, yet the literature remains largely classification-focused. In this light, we introduce a family of measures for total, aleatoric, and epistemic uncertainty based on proper scoring rules, with a particular emphasis on kernel scores. The framework unifies several well-known measures and provides a principled recipe for designing new ones whose behavior, such as tail sensitivity, robustness, and out-of-distribution responsiveness, is governed by the choice of kernel. We prove explicit correspondences between kernel-score characteristics and downstream behavior, yielding concrete design guidelines for task-specific measures. Extensive experiments demonstrate that these measures are effective in downstream tasks and reveal clear trade-offs among instantiations, including robustness and out-of-distribution detection performance.

Problem

• Developing kernel-based uncertainty measures for regression tasks
• Unifying existing uncertainty quantification methods through proper scoring rules
• Providing design guidelines for task-specific uncertainty measures

Innovation

• Unified framework using kernel scores for uncertainty
• Measures total, aleatoric, and epistemic uncertainty types
• Kernel choice governs robustness and detection performance