Search2Skill: Skill Distillation Beyond Knowledge Boundaries Via Rubric-Based Reinforcement Learning

2026-08-05

AI Summary

Current large language model agents are constrained by their parametric knowledge, limiting their ability to access domain-specific norms and standardized procedures beyond their training data. This work proposes an external evidence–based skill distillation framework that automatically identifies capability gaps, retrieves relevant external resources, and distills this evidence into structured, reusable expert skills. The framework further incorporates a rubric-driven reinforcement learning mechanism to jointly optimize the timing and modality of retrieval alongside the skill generation process. Evaluated across eight expert-level tasks spanning three benchmarks, the approach significantly outperforms existing retrieval-augmented and trajectory-based methods. The distilled skills demonstrate strong transferability across model scales, with performance gains attributable to skill abstraction rather than reliance on raw retrieved content.

Abstract

Reusable skills, which encapsulate the procedural knowledge required to solve real-world professional tasks, offer LLM-based agents a path toward self-evolution in expert domains. Existing self-evolving skill methods construct skills internally from the model's parametric knowledge or trajectories, and are therefore bounded by what the model already knows. However, the domain conventions and standard procedures underlying professional skills often lie beyond this boundary and are hard to elicit from the agent alone. To address this issue, we therefore propose a novel framework, Search2Skill, that automatically identifies the agent's capability gaps, searches external sources to address them, and distills the retrieved evidence into structured, reusable skills. Specifically, Search2Skill is optimized by a rubric-based reinforcement learning scheme that jointly improves when to search, how to search, and how to generate skills. Experiments on eight expert-level domains from three benchmarks show that Search2Skill consistently outperforms both search-augmented and trajectory-based skill-learning baselines under both streaming and held-out evaluation protocols. Further analyses show that the gains arise from skill abstraction rather than raw retrieved evidence, and that the acquired skills transfer across model scales.

Problem

• skill distillation
• knowledge boundaries
• professional tasks
• capability gaps
• external knowledge

Innovation

• skill distillation
• rubric-based reinforcement learning
• search-augmented reasoning
• capability gap identification
• structured skill abstraction