Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models

2026-09-22

AI Summary

研究使用层次贝叶斯模型评估大规模语言模型在不同难度问题上的能力和效率，发现能力随模型规模增加而提升但收益递减，效率则无明显改善。

Abstract

Capability and efficiency are two key dimensions of reasoning in large language models (LLMs). Capability refers to the ability to solve a given problem correctly, whereas efficiency refers to the ability to do so with limited resources. When LLMs use Chain-of-Thought (CoT) reasoning to solve problems of controlled hardness, both the number of problems solved correctly and the number of tokens required to reach a correct answer depend on problem hardness and model size. However, how these factors jointly shape capability and efficiency remains poorly understood. Here, we use hierarchical Bayesian models to evaluate the capability and efficiency of LLMs from the DeepSeek-R1-Distill model family across four classes of arithmetic and algorithmic reasoning problems. At a fixed model size, the probability of correctly solving an instance decays approximately exponentially with instance size, our proxy for problem hardness. The decay scale grows sublinearly with model size, indicating that larger models are more capable, but that capability gains diminish with scale. Output length grows as a power law with instance size, which serves as a proxy for difficulty. However, the parameters of this power law do not vary systematically with model size, suggesting that larger models do not become more efficient. Together, these findings reveal potential limitations of naive scaling as a strategy for developing more capable AI systems: capability improves with diminishing returns, while efficiency shows little to no improvement.

Problem

• Capability
• Efficiency
• Large Language Models
• Chain-of-Thought
• Problem Hardness

Innovation

• Hierarchical Bayesian Models
• Capability and Efficiency
• Chain-of-Thought (CoT) Reasoning
• Model Scaling