MagicSelector: Joint Optimization for Agent Tool Selection via Counterfactual Decomposition and Progressive Reranking

2026-07-20

AI Summary

This work addresses the challenges of ambiguous instruction interpretation, noise interference, and out-of-distribution (OOD) context confusion in agent-based tool retrieval by proposing a joint optimization framework. The approach leverages preference-guided counterfactual task decomposition to parse ambiguous instructions into atomic subtasks, combined with progressive re-ranking driven by self-distillation hard negative sampling and a semantic boundary-aware dynamic Top-K truncation strategy to achieve high-precision retrieval. Evaluated on the newly introduced MTDTool benchmark, the method significantly outperforms existing solutions, demonstrating substantial improvements in retrieval accuracy, OOD generalization, and token efficiency.

Abstract

We present MagicSelector, a joint optimization framework integrating Counterfactual task decomposition, Progressive reranking, and Dynamic Top-K, designed to address the fundamental challenges of tool retrieval in agents. MagicSelector is a specialized framework capable of translating ambiguous user instructions into executable atomic subtasks and guiding high-precision tool retrieval, effectively mitigating redundant noise and severe context distraction in out-of-domain (OOD) scenarios.We empower MagicSelector with these capabilities through three key contributions: (1) a preferenceguided counterfactual task decomposition mechanism that utilizes a counterfactual reward to quantify the marginal causal gain of decomposition on retrieval ranking, effectively imposing fine-grained structural supervision on logical coherence; (2) a progressive tool reranking method driven by self-distillation hard negative mining, which optimizes both point-wise and list-wise relevance to enhance fine-grained discrimination among highly similar tools; and (3) a dual semantic boundary-aware dynamic Top-K strategy that adaptively monitors reranking score cliffs and inter-tool semantic shifts to dynamically truncate the candidate list, maximizing relevant tool recall while filtering long-tail noise. Evaluated on MTDTool, the first task decomposition benchmark we constructed tailored for mobile multi-turn interactions with process-level annotations, MagicSelector yields promising performance. Extensive experiments demonstrate that MagicSelector significantly outperforms state-of-the-art methods in terms of tool retrieval accuracy, OOD generalization capability, and overall token efficiency, thereby demonstrating the effectiveness of our proposed framework.

Problem

• tool retrieval
• out-of-domain generalization
• task decomposition
• agent
• context distraction

Innovation

• Counterfactual Decomposition
• Progressive Reranking
• Dynamic Top-K
• Tool Retrieval
• OOD Generalization