Certainty Is Not Just Correctness: Rethinking Token-Level Certainty in LLM Reasoning

2026-09-26

AI Summary

This paper corrects the misconception of equating token-level certainty with correctness, revealing its limitations across different prediction targets. The study finds that in generated sequences, the certainty of early tokens reflects problem difficulty, whereas that of later tokens indicates answer correctness. Building on this insight, we propose a novel position-aware dynamic weighted voting mechanism to optimize test-time compute allocation strategies, validated through controlled empirical evaluations and cross-model analyses. Experimental results demonstrate that our approach improves accuracy from 78.71% to 79.54% while reducing generated token overhead by 82.4%, thereby achieving simultaneous gains in both inference precision and efficiency.

Abstract

Token-level certainty is widely used as a proxy for correctness in LLM training and inference. However, the performance of certainty-based methods depends both on the information in certainty scores and on how those scores are used. We therefore directly assess certainty's predictive ability through controlled empirical evaluations across models and tasks. We distinguish two prediction targets: identifying questions a model is more likely to answer correctly and distinguishing correct from incorrect responses to the same question. In our experiments, certainty is generally better at identifying questions a model is likely to answer correctly than at distinguishing correct from incorrect responses to the same question. Certainty also varies systematically across token types and positions within words, reflecting local properties of words and text form. Information about question difficulty appears early in generation, while the weaker information about answer correctness is more concentrated near the end. These findings show that the information certainty provides for decisions depends on the prediction target, the model, the certainty metric, and which token positions in the response are included in aggregation. We further demonstrate the practical value of these findings for test-time compute. We allocate the number of responses using certainty early in generation and weight answer votes using certainty near the end of each response. Compared with a fixed-sampling majority-voting baseline, this approach increases overall accuracy from 78.71\% to 79.54\% while reducing generated-token cost by 82.4\%.

Problem

• Token-level certainty
• LLM reasoning
• Correctness prediction
• Test-time compute

Innovation

• Token-level certainty
• LLM reasoning
• Test-time compute
• Prediction targets
• Majority voting