Predicting thinking time in Reasoning models

2025-06-29

AI Summary

To address degraded user experience caused by implicit chain-of-thought (CoT) reasoning and unpredictable response latency in large language models, this paper proposes the first “reasoning progress bar” framework specifically designed for hidden CoT. It supports both online real-time prediction and offline post-hoc analysis of reasoning duration. Methodologically, the framework integrates sequence modeling with behavioral trajectory analysis to extract fine-grained features from internal reasoning steps, enabling end-to-end latency prediction. Evaluated across diverse state-of-the-art reasoning paradigms—including Tree-of-Thought (ToT), Graph-of-Thought (GoT), and Self-Refine—the approach achieves an average prediction error below 12%, substantially improving response time predictability and mitigating user wait anxiety. This work pioneers the quantification and feedback of CoT latency as an interpretable, interactive signal, establishing a novel paradigm for designing trustworthy and controllable reasoning systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Reasoning under Uncertainty: Sequential Decision Making
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Reasoning models that produce long, hidden chains of thought have emerged as powerful tools for complex, reasoning-intensive taskscitep{deepseekai2025deepseekr1incentivizingreasoningcapability, openai2024openaio1card}. However, this paradigm introduces a new user experience challenge: users have little insight into how much time the model will spend reasoning before returning an answer. This unpredictability, can lead to user frustration and is likely to compound as LLMs can produce increasingly long tasks asynchronously citep{kwa2025measuringaiabilitycomplete}. In this paper, we introduce and evaluate methods for both online and offline prediction of model "thinking time," aiming to develop a practical "progress bar for reasoning." We discuss the implications for user interaction and future research directions.

Problem

• Predicting model thinking time in reasoning tasks
• Reducing user frustration from unpredictable response delays
• Developing progress indicators for hidden reasoning chains

Innovation

• Online and offline thinking time prediction
• Progress bar for reasoning models
• User interaction improvement techniques