MATCH: Model-Aware Tool Learning with Curriculum Scheduling and Hierarchically Gated Rewards

2026-09-17

AI Summary

为解决工具学习中固定阈值课程与策略能力不匹配及奖励泄漏问题，提出MATCH框架，采用自适应课程调度和层级门控奖励机制优化工具调用。

Abstract

Tool learning enables large language models (LLMs) to use external tools for tasks beyond parametric knowledge. Reinforcement learning can optimize tool-call behavior from feedback, but current methods still face two problems: fixed-threshold curricula can become misaligned with the policy's evolving capability boundary, and additive rewards can leak argument-level credit when the predicted tool is wrong. To address these problems, we propose MATCH, a closed-loop framework for model-aware tool learning with curriculum scheduling and hierarchically gated rewards. Model-Aware Curriculum Learning (MACL) maintains reward-derived sample difficulty that co-evolves with the policy, and each epoch selects samples near the current capability boundary together with a top-k pool of harder cases. Hierarchical Tool-call Gated Reward (HTGR) scores tool name, argument key, and argument value as a gated chain, granting credit at each level only when prerequisites hold. The same HTGR rewards drive both GRPO updates and MACL's difficulty refresh, closing the loop between policy optimization and sample scheduling. On API-Bank and BFCL V3, MATCH reaches 72.19% and 62.87% overall accuracy, outperforming the main supervised and RL-based baselines. Backbone experiments further show consistent improvements across four backbones from two model families.

Problem

• Tool Learning
• Reinforcement Learning
• Curriculum Scheduling
• Hierarchical Rewards

Innovation

• Model-Aware Curriculum Learning (MACL)
• Hierarchical Tool-call Gated Reward (HTGR)
• Curriculum Scheduling
• Hierarchically Gated Rewards