Teaching Language Models to Faithfully Express their Uncertainty

2025-10-14

AI Summary

Large language models (LLMs) often fail to faithfully express epistemic uncertainty in open-domain question answering—e.g., producing inconsistent answers to repeated queries without corresponding confidence signals—resulting in an “honesty gap.” To address this, we propose Faithful Uncertainty Tuning (FUT), a consistency-aware instruction-tuning method that requires no human annotation. FUT automatically constructs training data by analyzing answer consistency across self-generated samples and injects linguistic hedges (e.g., “possibly,” “likely”) and numerical confidence scores. It is decoder-agnostic, preserves the model’s original output distribution, and introduces negligible semantic shift. Experiments across multiple LLMs and benchmark datasets demonstrate that FUT significantly improves the faithfulness of uncertainty expression while maintaining baseline QA accuracy. This work establishes a lightweight, general-purpose, unsupervised paradigm for uncertainty calibration—advancing trustworthy AI.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large language models (LLMs) often miscommunicate their uncertainty: repeated queries can produce divergent answers, yet generated responses are typically unhedged or hedged in ways that do not reflect this variability. This conveys unfaithful information about the uncertain state of the LLMs' knowledge, creating a faithfulness gap that affects even strong LLMs. We introduce Faithful Uncertainty Tuning (FUT): a fine-tuning approach that teaches instruction-tuned LLMs to express uncertainty faithfully without altering their underlying answer distribution. We construct training data by augmenting model samples with uncertainty hedges (i.e. verbal cues such as 'possibly' or 'likely') aligned with sample consistency, requiring no supervision beyond the model and a set of prompts. We evaluate FUT on open-domain question answering (QA) across multiple models and datasets. Our results show that FUT substantially reduces the faithfulness gap, while preserving QA accuracy and introducing minimal semantic distribution shift. Further analyses demonstrate robustness across decoding strategies, choice of hedgers, and other forms of uncertainty expression (i.e. numerical). These findings establish FUT as a simple and effective way to teach LLMs to communicate uncertainty faithfully.

Problem

• LLMs miscommunicate their uncertainty in generated responses
• Unfaithful uncertainty expression creates a knowledge faithfulness gap
• Models need alignment between uncertainty hedges and answer variability

Innovation

• Fine-tuning approach for faithful uncertainty expression
• Augmenting model samples with uncertainty hedges
• Preserving answer accuracy while reducing faithfulness gap