uncertainty quantification

Designs and implements methods, models, and tools to estimate, represent, propagate, and visualize predictive uncertainty—including posterior, epistemic, and aleatoric components—using probabilistic (Bayesian, ensembles, conformal) and non‑probabilistic techniques, as well as online and post‑hoc estimators and uncertainty sampling. Builds evaluation and calibration procedures, metrics and benchmarks to quantify coverage and reliability (e.g., nonconformity scores, calibration and weighting schemes), to flag or abstain on high‑uncertainty cases, and to combine or aggregate multiple uncertainty sources.

uncertaintyquantification

Recent Skill Trend

Momentum and market value over time
Trending
Score
No comparison yet
1.99
Oct 01, 2026Oct 01, 2026
Career
Value
No comparison yet
$205K/year
Oct 01, 2026Oct 01, 2026

Must-Read Papers

Most classic and influential ideas
View more

From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation

Feb 16, 2024
NK
Nikita Kotelevskii
🏛️ CAIT | Skoltech | MBZUAI

This work addresses the challenge of entangled uncertainty sources and the difficulty of disentangling pointwise statistical risk in predictive modeling. We propose a unified generative framework based on approximate Bayesian inference that, for the first time, establishes an explicit, interpretable decomposition linking pointwise statistical risk to two fundamental uncertainty types: aleatoric uncertainty (arising from inherent data noise) and epistemic uncertainty (stemming from model ignorance). The framework jointly generates multiple uncertainty measures while ensuring semantic consistency across them. Experiments on image benchmarks demonstrate significant improvements in out-of-distribution detection and misclassification identification, achieving higher AUROC scores compared to existing methods. Our approach thus provides robust, quantifiable uncertainty estimates essential for downstream uncertainty-aware tasks such as active learning, safe decision-making, and model debugging.

Decomposes statistical pointwise riskGenerates predictive uncertainty measuresValidates method on image datasets

This study addresses the computational bottleneck in evaluating the calibration of nested uncertainty sets within expensive simulation models. To overcome this limitation, the work proposes an efficient calibration assessment method grounded in a Bayesian framework and the Dirichlet-Multinomial model. By exploiting the nested structure, the approach directly processes interval outputs without requiring access to the full predictive distribution. Furthermore, it incorporates Bayes factor testing for statistical inference, substantially reducing the number of independent simulations needed. The proposed method successfully detects model miscalibration in data assimilation tasks under limited simulation budgets. Overall, this work significantly lowers computational costs while demonstrating both the effectiveness and practical utility of the proposed approach for calibrating complex simulation systems.

Calibration assessmentComputational costCoverage probability

This study addresses the quantification of sources of predictive uncertainty and their contributions to prediction interval width. Building upon the law of total variance, the work proposes several conservative decompositions of posterior predictive variance, systematically characterizing the components of uncertainty and their interdependencies through conditional expectation and conditional variance terms. Experimental evaluations across multiple canonical models demonstrate that the proposed approach effectively identifies the dominant sources of uncertainty and reveals coherent patterns of co-variation among decomposition terms. These insights offer a novel perspective for model assessment and refinement, enhancing interpretability and guiding targeted improvements in predictive reliability.

Law of Total VariancePosterior Predictive VariancePrediction Intervals

Latest Papers

What's happening recently
View more

This study addresses the persistent challenges of inadequate statistical coverage and inaccurate uncertainty quantification in existing AI-based weather forecasting models, particularly during extreme events. It introduces, for the first time, an online conformal prediction framework that makes no distributional assumptions to post-process outputs from three leading global probabilistic AI models—GenCast, NeuralGCM, and AIFS-ENS. The proposed method significantly improves the statistical coverage accuracy of temperature and precipitation forecasts, including extreme events, without compromising other probabilistic performance metrics. By providing mathematically rigorous uncertainty guarantees, this approach enables reliably calibrated AI-driven weather predictions.

calibrationextreme eventsprobabilistic weather forecasting

This study addresses the common conflation of epistemic uncertainty (EU) and aleatoric uncertainty (AU) in existing wind power forecasting methods, which undermines decision reliability. Building upon the law of total variance, the authors propose a unified framework combining heteroscedastic neural networks with Bayesian posterior approximation to explicitly disentangle total predictive uncertainty into its AU and EU components. A β-negative log-likelihood (β-NLL) loss is introduced to balance mean and variance learning. The work presents the first evaluation framework for uncertainty disentanglement in wind power prediction that operates without ground-truth uncertainty labels. Through synthetic experiments, data attribute analysis, and scaling studies, the approach demonstrates theoretically consistent responses of AU and EU to noise structure, distribution shifts, and training set size on both synthetic and real-world SCADA data, validating its theoretical soundness and practical utility.

aleatoric uncertaintyepistemic uncertaintyuncertainty quantification

This study addresses the challenges of inaccurate uncertainty quantification and insufficient coverage of confidence sets in predictive resampling by proposing a bagged martingale posterior method. This approach integrates Bayesian inference, bootstrap techniques, and high-dimensional quantile modeling to achieve conservative calibration by initiating predictive paths from random bootstrap samples and aggregating the results. Its core innovation lies in substantially improving calibration accuracy without additional simulation costs, thereby overcoming the reliance of conventional methods on algorithmic tuning. Experimental results demonstrate that the proposed method delivers computationally efficient and conservatively calibrated uncertainty quantification in complex scenarios, such as sparse high-dimensional regression.

CalibrationCredible setsMartingale posteriors

Hot Scholars

FC

Fabio Cuzzolin

Professor of Artificial Intelligence, Oxford Brookes University
Artificial IntelligenceImprecise ProbabilitiesBelief FunctionsComputer Vision
EH

Eyke Hüllermeier

Professor of Computer Science, Paderborn University
Artificial IntelligenceMachine LearningFuzzy LogicBioinformatics
MV

Matias Valdenegro-Toro

Assistant Professor of Machine Learning, Bernoulli Institute, University of Groningen
Uncertainty in Machine LearningBayesian Deep LearningRobot Perception
MP

Maxim Panov

Assistant Professor at Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)
Machine LearningStatistics
MC

Michele Caprio

Lecturer (Asst. Prof.), The University of Manchester
Imprecise ProbabilityApplied ProbabilityArtificial IntelligenceStatistical Theory