On the Token Value Inequality in Efficient Reasoning

2026-09-27

AI Summary

This study addresses the bottleneck of excessive token consumption and uneven value distribution in Chain-of-Thought (CoT) reasoning. To this end, it proposes TokenProbe, a framework that leverages log-probability signals to precisely identify core and redundant tokens, enabling efficient inference through selective compression. Furthermore, by revealing the non-uniformity of token value, the authors design an efficient GRPO-based reinforcement learning objective that utilizes core tokens as proxies for optimization. The proposed framework reduces token usage by 76% while preserving reasoning quality, and surpasses flagship baseline models under equivalent computational budgets. Ultimately, this work establishes a novel paradigm for efficient CoT reasoning in large language models.

Abstract

Chain-of-Thought reasoning has enabled large language models to achieve substantial performance gains on complex tasks. However, these gains come at the cost of dramatically increased token consumption. This raises a fundamental question: is every token in the reasoning trace equally valuable? We present a diagnostic and optimization framework grounded in a key empirical finding: the value of tokens within a CoT reasoning sequence is highly non-uniform, and this non-uniformity can be effectively characterized by token-level log probability signals. We show that normalized log probability helps distinguish core tokens, which carry structural and decisive reasoning content, from redundant tokens, which are exploratory, low-confidence filler that contributes less directly to the final answer. Building on these findings, we formulate the TokenProbe framework around two empirical findings and one claim: findings identify token value inequality first and then establish TokenProbe as a core-token proxy, and the claim introduces an efficient GRPO objective positing that selectively compressing redundant tokens can yield Pareto improvements in the accuracy-token efficiency space. Empirically, our method preserves reasoning quality while reducing the token usage by 76% of the baseline. Under matched reasoning-length budgets, we show that it can even outperform strong flagship baselines like Gemini-3.1-Pro. Homepage: https://runjia.tech/tokenprobe/.

Problem

• Chain-of-Thought reasoning
• Token efficiency
• Token value inequality
• Large language models

Innovation

• Chain-of-Thought
• Token Value Inequality
• TokenProbe
• GRPO
• Log Probability