Adapting Embedding Models for Agent Capability Retrieval

2026-07-19

AI Summary

This work addresses the challenge of accurately retrieving executable capabilities from heterogeneous agent marketplaces that comprise native agents, toolkits, and reusable skills. It presents the first adaptation of general-purpose text embedding models—BGE-base, KaLM-v1.5, and EasyRec—to this setting by constructing capability profiles from publicly available metadata and applying supervised fine-tuning on the AgentSelect benchmark. The approach is evaluated for its cross-catalog generalization ability, demonstrating significant performance gains on both the MuleRun native agent benchmark and the ClawHub skill benchmark, which includes 50 skills and 1,000 queries. These results validate the effectiveness and transferability of the proposed fine-tuning strategy for capability retrieval in mixed-agent environments.

Abstract

Open agent marketplaces list native agents, tool bundles, and reusable skill packages in the same search interface, yet practitioners still have little guidance on how to retrieve across this mixed catalog. We study whether off-the-shelf retrieval models, trained for general text retrieval, can be adapted to match user queries to executable agent capabilities, and whether the learned signal transfers beyond the benchmark used for tuning. We fine-tune three open retrieval backbones, BGE-base, KaLM-v1.5, and EasyRec, on AgentSelect, which represents marketplace-visible units as capability profiles derived from public metadata, and test transfer on two catalogs not seen during training: MuleRun native agents and a ClawHub benchmark of 50 skills with 1,000 queries. Adaptation helps on both catalogs. Code and data will be released upon publication.

Problem

• agent capability retrieval
• open agent marketplace
• mixed catalog search
• executable agent capabilities
• retrieval adaptation

Innovation

• agent capability retrieval
• retrieval model adaptation
• capability profiling
• cross-catalog transfer
• fine-tuning