Retrieval-Augmented Skill Optimization via Cross-Harness Adaptation

2026-09-29

AI Summary

This study addresses the limitation that existing agent skill optimization overlooks knowledge from public skill repositories and excessively relies on costly trial-and-error. To this end, we propose RASO, a Retrieval-Augmented Skill Optimization framework that leverages external skill corpora as priors. RASO introduces a novel retrieval-augmented mechanism integrated with cross-tool adaptation, achieving skill initialization through Retrieval-Augmented Generation (RAG) and cross-domain adaptation algorithms. It further refines skills iteratively based on execution feedback, enabling the construction of high-quality skills without initial trial-and-error. Experiments conducted across four benchmarks and two large language models demonstrate that RASO consistently outperforms non-retrieval baselines, substantially reducing optimization costs while significantly enhancing task performance.

Abstract

An agent skill is a reusable, actionable natural-language artifact that guides an agent to perform a task effectively under a given harness. Recent studies have explored the optimization of agent skills, contributing to a growing collection of publicly available skills spanning diverse tasks, domains, and harnesses. Despite millions of publicly shared skills, existing skill optimization methods largely overlook this accumulated knowledge, instead relying solely on expensive agent rollouts to iteratively refine skills for a target task. To address this, we propose \textbf{Retrieval-Augmented Skill Optimization (RASO)}, a framework that leverages an external skill corpus as prior knowledge throughout skill optimization. RASO retrieves relevant knowledge from existing skills and adapts it to the target task and harness via Cross-Harness Adaptation, accounting for mismatches in both domain and harness. RASO comprises two complementary stages: \textbf{Retrieval-Augmented Skill Initialization (RASI)} constructs a knowledge-grounded initial skill without requiring agent rollouts, while \textbf{Retrieval-Augmented Skill Update (RASU)} iteratively refines the skill by retrieving external knowledge guided by execution feedback. Across four agent benchmarks and two models, extensive experiments show that RASO consistently outperforms baselines without retrieval-augmented skill initialization and updating.

Problem

• Skill Optimization
• Retrieval Augmentation
• Cross-Harness Adaptation
• Agent Skills

Innovation

• Retrieval-Augmented Skill Optimization
• Cross-Harness Adaptation
• Agent Skills
• Skill Initialization
• Execution Feedback