Tools Are Not Islands: Set-Level Tool Retrieval for LLM Agents via Query-Conditioned Hyperedge Prediction

2026-07-28

AI Summary

This work addresses a key limitation in existing tool retrieval methods for large language model agents, which typically evaluate individual tools or sequential combinations in isolation and thus fail to capture the collective utility of entire toolsets. The authors reformulate tool retrieval as a query-conditioned hyperedge prediction problem on a tool co-invocation hypergraph, enabling joint scoring of complete toolsets for the first time. Their approach incorporates cardinality-aware compatibility modeling and subsumes prevailing retrieval paradigms as special cases without requiring modifications to downstream agents. It further supports zero-shot and few-shot generalization across both tools and domains. Evaluated on ToolBench, the proposed method significantly outperforms state-of-the-art baselines, yielding substantial improvements in both tool retrieval accuracy and end-to-end task success rates.

Abstract

Large language model (LLM) agents increasingly rely on invoking external tools to complete real-world tasks. Tool retrieval, which selects a small task-relevant subset from a library of thousands of tools before the agent acts, has therefore become a critical component of LLM agent pipelines. However, existing retrievers either score each tool in isolation or assemble the tool set sequentially, so the joint utility of a candidate set is never evaluated as a whole. In this paper, we propose HYSET, short for HYperedge-based SEt-level Tool retrieval. Our contributions are threefold: (i) we formulate tool retrieval as query-conditioned hyperedge prediction on a tool co-invocation hypergraph, under which the tool set itself becomes the unit of scoring and most existing retrieval paradigms reduce to restricted instances; (ii) we capture size-dependent tool compatibility through cardinality-specific interactions; and (iii) we design HYSET as a pre-selection module requiring no modification to the downstream agent. Experiments on ToolBench demonstrate that HYSET consistently outperforms state-of-the-art baselines in both tool retrieval performance and end-to-end task success. Beyond the in-domain setting, HYSET further supports zero-shot/few-shot transfer, generalizing to held-out tools/categories and unseen domains with minimal supervision.

Problem

• tool retrieval
• LLM agents
• set-level selection
• hyperedge prediction
• tool co-invocation

Innovation

• set-level retrieval
• hyperedge prediction
• tool co-invocation
• cardinality-specific interaction
• LLM agents