Quantifying Faithful Confidence Expression in Large Reasoning Models

2026-06-02

AI Summary

This study addresses the prevalent misalignment between linguistic confidence and intrinsic uncertainty in large reasoning models (LRMs), which manifests as insufficient faithfulness of confidence (FC) and undermines model trustworthiness. The work proposes the first generalizable FC evaluation framework tailored for long-chain reasoning, leveraging prefix-conditioned sampling to control the structure and conditional variability of reasoning trajectories. It systematically quantifies the relationship between linguistic assertiveness and three types of internal uncertainty through intrinsic confidence estimators based on token probabilities, hidden states, and response consistency. Experimental results demonstrate that LRMs generally exhibit inadequate FC, that the reasoning process itself does not inherently improve FC, and that prompting interventions effective for non-reasoning models fail in reasoning contexts.

Abstract

Reliable uncertainty communication is critical to the trustworthiness of LLMs, yet faithful calibration (FC)--the alignment between models' intrinsic and (linguistically) expressed confidence--is a persistent failure mode. This challenge is key for large reasoning models (LRMs), whose extended reasoning traces are often interpreted by users as evidence of deliberation, competence, and confidence. Despite the importance of FC and wide usage of LRMs, the extent to which LRMs can faithfully express their confidence remains poorly understood. Moreover, the prevailing paradigm to measure FC does not generalize well to the long chain-of-thought outputs generated by LRMs, which tend to lack clear step boundaries, involve inconsistent step structure, and encode complex conditional dependencies throughout the trace--complicating estimation of intrinsic confidence. To address this challenge, we introduce a novel framework to systematically quantify FC of LRMs. Our framework analyzes linguistic decisiveness relative to three sources of internal uncertainty, based on token probabilities, hidden states, and sampled response consistency. We also devise a prefix-conditioned sampling approach to control for conditional and structural variation across traces. Applying our framework to a diverse suite of leading models, datasets, and prompts, we find that faithful confidence expression is a significant challenge for LRMs. Reasoning behaviors do not automatically translate to improved FC, and prompt interventions for non-reasoning models do not improve faithfulness in the reasoning setting. Different confidence estimators further produce divergent assessments of the same traces, revealing fragility in prior evaluation methodologies. Taken together, our work establishes FC as a distinct reliability and alignment target for LRMs, particularly as such systems are increasingly deployed in high-stakes contexts.

Problem

• faithful calibration
• large reasoning models
• uncertainty communication
• confidence expression
• chain-of-thought reasoning

Innovation

• faithful calibration
• large reasoning models
• uncertainty quantification
• chain-of-thought reasoning
• confidence expression