UbiQTree: Uncertainty Quantification in XAI with Tree Ensembles

2025-08-13

AI Summary

Current SHAP explanation methods treat feature importance as deterministic point estimates, neglecting inherent aleatoric (data-driven) and epistemic (model-driven) uncertainties in both predictive models and input data—limiting their trustworthy deployment in high-stakes domains such as healthcare. To address this, we propose the first framework integrating Dempster–Shafer evidence theory with Dirichlet process-based hypothesis sampling, enabling decoupled quantification of three intertwined uncertainty sources in SHAP values: aleatoric, epistemic, and feature interaction uncertainty. Evaluated on tree ensemble models, our approach reveals that the stability of high-magnitude SHAP values is strongly governed by epistemic uncertainty—a limitation mitigable via data augmentation and model calibration. Empirical validation across multiple real-world clinical datasets demonstrates significant improvements in explanation robustness and decision reliability, establishing a foundation for uncertainty-aware, clinically actionable model interpretation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Security and Privacy: Data transparency and provenance

Abstract

Explainable Artificial Intelligence (XAI) techniques, such as SHapley Additive exPlanations (SHAP), have become essential tools for interpreting complex ensemble tree-based models, especially in high-stakes domains such as healthcare analytics. However, SHAP values are usually treated as point estimates, which disregards the inherent and ubiquitous uncertainty in predictive models and data. This uncertainty has two primary sources: aleatoric and epistemic. The aleatoric uncertainty, which reflects the irreducible noise in the data. The epistemic uncertainty, which arises from a lack of data. In this work, we propose an approach for decomposing uncertainty in SHAP values into aleatoric, epistemic, and entanglement components. This approach integrates Dempster-Shafer evidence theory and hypothesis sampling via Dirichlet processes over tree ensembles. We validate the method across three real-world use cases with descriptive statistical analyses that provide insight into the nature of epistemic uncertainty embedded in SHAP explanations. The experimentations enable to provide more comprehensive understanding of the reliability and interpretability of SHAP-based attributions. This understanding can guide the development of robust decision-making processes and the refinement of models in high-stakes applications. Through our experiments with multiple datasets, we concluded that features with the highest SHAP values are not necessarily the most stable. This epistemic uncertainty can be reduced through better, more representative data and following appropriate or case-desired model development techniques. Tree-based models, especially bagging, facilitate the effective quantification of epistemic uncertainty.

Problem

• Quantifying uncertainty in SHAP values for tree ensembles
• Decomposing SHAP uncertainty into aleatoric and epistemic components
• Assessing reliability of feature attributions in high-stakes applications

Innovation

• Decomposes SHAP uncertainty into aleatoric and epistemic components
• Integrates Dempster-Shafer theory with Dirichlet process sampling
• Uses tree ensembles for effective epistemic uncertainty quantification