Uncertainty Estimation using Variance-Gated Distributions

2025-09-07

AI Summary

Accurately quantifying sample-wise uncertainty in neural networks for high-stakes applications remains challenging, particularly in disentangling epistemic from aleatoric uncertainty—existing additive decomposition methods often fail to do so reliably. Method: We propose a novel uncertainty estimation and decomposition framework grounded in the signal-to-noise ratio (SNR) of class probability distributions. It introduces a variance-gating mechanism to dynamically model prediction reliability and an ensemble confidence factor to adaptively scale probabilistic outputs. Contribution/Results: Our approach reinterprets the “committee diversity collapse” phenomenon and overcomes fundamental limitations of additive decomposition, enabling finer-grained, well-calibrated separation of epistemic and aleatoric uncertainty. Extensive evaluation across multiple benchmark datasets demonstrates superior discriminability and calibration of uncertainty estimates. The framework advances both theoretical understanding and practical deployment of trustworthy AI systems.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Evaluation of per-sample uncertainty quantification from neural networks is essential for decision-making involving high-risk applications. A common approach is to use the predictive distribution from Bayesian or approximation models and decompose the corresponding predictive uncertainty into epistemic (model-related) and aleatoric (data-related) components. However, additive decomposition has recently been questioned. In this work, we propose an intuitive framework for uncertainty estimation and decomposition based on the signal-to-noise ratio of class probability distributions across different model predictions. We introduce a variance-gated measure that scales predictions by a confidence factor derived from ensembles. We use this measure to discuss the existence of a collapse in the diversity of committee machines.

Problem

• Evaluating per-sample uncertainty quantification in neural networks
• Decomposing predictive uncertainty into epistemic and aleatoric components
• Proposing intuitive framework using signal-to-noise ratio of distributions

Innovation

• Variance-gated confidence scaling method
• Signal-to-noise ratio based decomposition framework
• Ensemble-derived uncertainty estimation technique