Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models

2026-09-22

AI Summary

通过API诱导前沿模型外部化中间推理过程，验证其推理能力，并分析不同模型的推理结构和效率。

Abstract

The rapid capability gains of frontier language models are widely attributed to improved reasoning abilities, yet this cannot be verified as raw CoT traces in closed-source systems are hidden. By registering a simple custom tool through a standard API feature, we induce frontier models to externalize intermediate reasoning. Because these traces may reflect post-hoc rationalization rather than genuine reasoning, we first evaluate against native CoT on open-source models and extend to closed-source frontier models including GPT-6 Astra. We find that the extracted reasoning matches native reasoning performance and substantially outperforms no-reasoning baselines, across competition mathematics, science, and code generation. We then characterize how frontier models structure their intermediate reasoning. Across token efficiency, reasoning-step types, and induced reasoning trees, we identify systematic differences in how models externalize, compress, and organize reasoning. We find that Astra exhibits token-efficient directed reasoning, selecting a correct trajectory earlier, while resolving elementary steps internally and externalizing only crucial reasoning. These findings provide a behavioral lens on frontier-model reasoning beyond benchmark scores.

Problem

• frontier language models
• reasoning abilities
• chain-of-thought
• closed-source systems

Innovation

• Custom Tool Induction
• Externalized Reasoning
• Token Efficiency
• Intermediate Reasoning Structure
• GPT-6 Astra