🤖 AI Summary
This study addresses the implications of frontier AI models performing complex reasoning without explicit chain-of-thought (CoT) prompting, which could undermine CoT-dependent safety alignment mechanisms. The authors present the first systematic evaluation across over 30,000 problems spanning 43 benchmarks—including mathematics, programming, and causal reasoning—and introduce quantifiable thresholds for CoT-free reasoning: the 50% task completion time threshold (TH) and a corresponding reasoning token threshold, establishing metrics comparable to human reasoning efficiency. Their findings reveal that the 50% TH for leading models approximately doubles annually, with GPT-5.5 already exceeding three minutes; projections indicate thresholds surpassing seven minutes by 2028 and 25 minutes by 2030, highlighting the rapid advancement of CoT-free reasoning capabilities and its significant safety implications.
📝 Abstract
Many efforts to ensure frontier AI models are safe rely on monitoring their chain-of-thought (CoT) reasoning. If models become able to perform sufficiently complex reasoning internally, without explicit thinking tokens, this would undermine such oversight. We measure how well frontier models reason without CoT across a suite of over 30,000 questions spanning 43 benchmarks in domains including math, coding, puzzles, causality, theory-of-mind, and strategic reasoning. To compare models against humans, we estimate the $50\%$-task-completion time horizon (TH): the human time required for tasks a model completes with $50\%$ success rate. We complement this with a $50\%$ reasoning token horizon: the minimum number of o3-mini reasoning tokens needed for tasks a model solves with $50\%$ success rate. We find that the no-CoT $50\%$ TH of frontier models has been doubling roughly every year over the past six years, with GPT-5.5's TH reaching over 3 minutes and reasoning token horizon exceeding 1,500 tokens. Our median estimates predict that frontier no-CoT THs could exceed 7 minutes by 2028, and 25 minutes by 2030, though these projections carry substantial uncertainty. We recommend frontier developers track this explicitly.