Technology Category
Machine Learning: Calibration & Uncertainty QuantificationData Mining & Knowledge Management: Anomaly/Outlier DetectionReasoning under Uncertainty: Relational Probabilistic Models
A Simple and Effective Method for Uncertainty Quantification and OOD Detection
🤖 AI Summary
To address the high computational and storage overhead of Bayesian neural networks and deep ensembles in uncertainty quantification and out-of-distribution (OOD) detection, this paper proposes an efficient OOD detection method based on density modeling in the feature space of a single deterministic model. The core innovation lies in constructing an information potential field over deep features using kernel density estimation (KDE), explicitly characterizing the density distribution of training data in the latent space—enabling rapid, sampling-free, and ensemble-free uncertainty estimation. Our approach requires only a single pre-trained deterministic network, drastically reducing inference latency and memory footprint. Evaluated on benchmark tasks—including Two Moons, Three Spirals, and CIFAR-10 vs. SVHN—the method achieves state-of-the-art performance in standard metrics such as FPR95 and AUROC, demonstrating superior accuracy, robustness, and practical deployability.
Technology Category
Application Category
📝 Abstract
Bayesian neural networks and deep ensemble methods have been proposed for uncertainty quantification; however, they are computationally intensive and require large storage. By utilizing a single deterministic model, we can solve the above issue. We propose an effective method based on feature space density to quantify uncertainty for distributional shifts and out-of-distribution (OOD) detection. Specifically, we leverage the information potential field derived from kernel density estimation to approximate the feature space density of the training set. By comparing this density with the feature space representation of test samples, we can effectively determine whether a distributional shift has occurred. Experiments were conducted on a 2D synthetic dataset (Two Moons and Three Spirals) as well as an OOD detection task (CIFAR-10 vs. SVHN). The results demonstrate that our method outperforms baseline models.
Problem
Research questions and friction points this paper is trying to address.
Quantify uncertainty in deterministic models efficiently
Detect out-of-distribution samples using feature space density
Reduce computational cost compared to Bayesian neural networks
Innovation
Methods, ideas, or system contributions that make the work stand out.
Single deterministic model for uncertainty quantification
Feature space density using kernel estimation
Effective OOD detection via density comparison
🔎 Similar Papers
No similar papers found.
