🤖 AI Summary
This work addresses the inefficiency of large language models in tool-integrated reasoning, where redundant external tool invocations often occur due to an inability to assess task difficulty. To mitigate this, we propose AdaTIR, a framework that employs a reinforcement learning–driven, difficulty-aware strategy to dynamically allocate tool-call budgets: performing internal reasoning for simple tasks and selectively invoking tools for complex ones. AdaTIR introduces a novel difficulty-aware efficiency reward and Clipped Advantage Shaping (CAS) to alleviate the suppression of correctness rewards by tool penalties, thereby achieving an adaptive balance between internal reasoning and tool usage. Experiments show that AdaTIR reduces tool calls by 97.6% on simple tasks and 28.2% on complex tasks while maintaining or improving accuracy; notably, it outperforms baselines by 4.8% on the AIME 2024 no-tool evaluation.
📝 Abstract
Tool-Integrated Reasoning (TIR) has significantly enhanced the capabilities of Large Language Models (LLMs), yet current agents tend to exhibit cognitive offloading, redundantly invoking external tools even for simple tasks. In this paper, we suggest that true agentic intelligence requires not just tool invocation, but the adaptive wisdom to discern when to use them. We propose AdaTIR, a framework that shifts the paradigm from static tool invocation to difficulty-aware reasoning internalization. By introducing a difficulty-aware efficiency reward, AdaTIR dynamically adjusts tool budgets based on task complexity--internalizing reasoning for simple tasks while selectively invoking tools for complex tasks. Furthermore, we identify a sign reversal problem where tool penalties outweigh correctness rewards, mistakenly penalizing correct rollouts with negative advantages. To resolve this, we propose Clipped Advantage Shaping (CAS), which ensures that correctness remains the primary objective while using efficiency as a secondary constraint. Empirical results demonstrate that AdaTIR reduces tool calls by up to 97.6% on simple tasks and 28.2% on complex challenges while maintaining or enhancing accuracy. Notably, AdaTIR successfully internalizes reasoning, outperforming baselines by 4.8% on AIME 2024 even when tool access is strictly disabled.