Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning

2026-07-31

AI Summary

Large language models often generate seemingly plausible yet semantically vacuous or erroneous reasoning on tasks beyond their capabilities, thereby misleading users. This work presents the first systematic characterization of such “invalid reasoning” and introduces Capability-aligned Reinforcement Learning (CaRL), a novel approach that aligns model behavior with its actual reasoning capacity. CaRL leverages reward shaping and post-hoc refusal augmentation to transform failed inference attempts into supervision signals for abstention, encouraging the model to proactively decline answering when effective reasoning is unattainable. Empirical results demonstrate that this method substantially reduces invalid reasoning across diverse task difficulties while preserving original performance on solvable problems, achieving effective alignment between the model’s capabilities and its output behavior.

Abstract

Large language models generate computationally expensive yet semantically void reasoning on beyond-capability tasks, creating risks where plausible-sounding but incorrect derivations mislead users. We characterize this \textit{futile reasoning} phenomenon through systematic analysis, revealing universal capability overreach and systematic miscalibration between capability and behavior. The dominant failure mode is specious reasoning, which outputs look superficially valid but contain subtle errors, escalating with task difficulty. To address this, we introduce \textbf{CaRL} (\textbf{Ca}pability-\textbf{a}ligned \textbf{R}einforcement \textbf{L}earning), which aligns model behavior with capability boundaries through reward shaping that incentivizes refusal over futile reasoning and hindsight refusal augmentation that converts failures into refusal supervision. Experiments demonstrate a substantial reduction in futile reasoning while preserving performance across task difficulties, effectively achieving capability-aligned behavior without sacrificing utility. \footnote{https://github.com/icip-cas/Knowing-When-to-Quit}

Problem

• futile reasoning
• capability overreach
• specious reasoning
• miscalibration
• large language models

Innovation

• futile reasoning
• capability alignment
• refusal behavior
• reward shaping
• hindsight refusal augmentation