TrAC: Trace-Conditioned Answer Consistency for Efficient Uncertainty Quantification in LLMs

2026-07-31

AI Summary

This work addresses the challenge of response-level uncertainty estimation for reasoning outputs from large language models (LLMs), which are often compromised by hallucination-induced errors. The authors propose a unified uncertainty quantification framework that integrates active and passive signals: Prefix-Conditioned Excitation (PCE) actively re-excites answers from a single completed reasoning trace to assess consistency, while Trace Uncertainty Profiling (TUP)—requiring no additional decoding—characterizes the evolution of uncertainty throughout the reasoning process. A lightweight prediction head jointly models these signals. Evaluated across five mathematical reasoning benchmarks and three LLM families, the method outperforms the 8-sample self-consistency baseline by 1.8% in macro AUROC and reduces AURC by 3.4%; when reusing existing 8-sample ensembles, it further improves AUROC by 4.3% and lowers AURC by 8.3%.

Abstract

Large language models (LLMs) can generate fluent reasoning traces that nevertheless lead to incorrect answers, making response-level uncertainty estimation important for abstention, human review, and adaptive compute allocation. Existing approaches generally fall into three categories: passive single-trace methods use token-level confidence signals, sampling-based methods compare multiple complete traces at higher generation cost, and active prefix-based methods probe partial traces to study answer stabilization or preference transitions. However, none actively re-elicits an answer from a completed reasoning trace to measure its consistency with and support for the original answer. To address this gap, we introduce Trace-Conditioned Answer Consistency (TrAC), a correctness-supervised uncertainty quantification framework that combines active and passive signals anchored to one completed reasoning trace. Its active component, Prefix-Conditioned Elicitation (PCE), re-elicits a short answer conditioned on the completed trace and represents both its consistency with the original answer and its token-level probabilistic support. Its passive component, Trace Uncertainty Profile (TUP), summarizes how token-level uncertainty evolves throughout the original generation without additional decoding. A lightweight head then integrates the two representations into a response-correctness score. Across five mathematical reasoning benchmarks and three LLM families, TrAC improves macro AUROC by 1.8% and reduces AURC by 3.4% relative to eight-sample self-consistency, while using one complete reasoning trace and a short cached answer probe. When eight samples are already available, augmenting sample consensus with re-elicitation further improves macro AUROC by 4.3% and reduces AURC by 8.3%, without additional full-trace generation.

Problem

• uncertainty quantification
• large language models
• answer consistency
• reasoning traces
• abstention

Innovation

• Trace-Conditioned Answer Consistency
• Uncertainty Quantification
• Reasoning Traces
• Prefix-Conditioned Elicitation
• Large Language Models