Conceptualizing Uncertainty

2025-03-05

AI Summary

To address the challenge of interpreting model uncertainty in high-dimensional classification tasks, this paper proposes the first Concept Activation Vector (CAV)-based framework for uncertainty explanation. Unlike conventional eXplainable AI (XAI) methods that provide only local feature attributions, our framework traces uncertainty sources to human-interpretable semantic concepts, enabling both local attribution and global semantic analysis of uncertainty. Furthermore, by embedding uncertainty explanations into a model self-feedback loop, the framework supports calibration-aware model optimization. Extensive experiments across multiple high-dimensional benchmark datasets demonstrate that our approach significantly improves the semantic consistency and credibility of uncertainty explanations, while effectively guiding uncertainty-aware model calibration. This work bridges the gap between uncertainty quantification and semantic interpretability, offering a principled, concept-level lens for diagnosing and refining uncertain predictions in complex classification systems.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Uncertainty in machine learning refers to the degree of confidence or lack thereof in a model's predictions. While uncertainty quantification methods exist, explanations of uncertainty, especially in high-dimensional settings, remain an open challenge. Existing work focuses on feature attribution approaches which are restricted to local explanations. Understanding uncertainty, its origins, and characteristics on a global scale is crucial for enhancing interpretability and trust in a model's predictions. In this work, we propose to explain the uncertainty in high-dimensional data classification settings by means of concept activation vectors which give rise to local and global explanations of uncertainty. We demonstrate the utility of the generated explanations by leveraging them to refine and improve our model.

Problem

• Explaining uncertainty in high-dimensional machine learning predictions.
• Developing global explanations for model uncertainty using concept activation vectors.
• Enhancing model interpretability and trust through refined uncertainty explanations.

Innovation

• Concept activation vectors explain uncertainty.
• Local and global uncertainty explanations provided.
• High-dimensional data classification enhanced.