skill library management

Existing benchmarks struggle to evaluate agents’ ability to abstract and reuse high-level tool compositions—referred to as “skills”—over long-horizon tasks. To address this gap, this work proposes SkillCraft, a novel benchmark that explicitly centers on skill formation and cross-task reuse as core evaluation dimensions. SkillCraft features highly compositional and scalable real-world tool-use scenarios, accompanied by a lightweight evaluation protocol. The framework enables large language model agents to automatically compose atomic tools into executable skills, cache them both within and across tasks, and build a persistent skill library. Experiments demonstrate that advanced agents leveraging skill reuse reduce token consumption by up to 80%, with task success rates showing strong positive correlation with their runtime tool-composition capabilities.

This study addresses the current lack of systematic understanding of reusable agent skills in software engineering, particularly regarding their coverage across the software lifecycle. For the first time, it adopts an activity-oriented perspective and conducts a large-scale empirical investigation to collect, categorize, and model software engineering skills from public skill repositories. The work systematically characterizes the types of encapsulated activities, their evolutionary patterns, and evaluation mechanisms. Findings reveal that engineering activities with high contextual dependency are progressively being transformed into reusable skills. Building on these insights, the paper outlines promising future directions, including skill recommendation, structured organization of skill repositories, and enhanced encapsulation strategies for high-context skills, thereby providing both theoretical foundations and practical guidance for agent-driven software engineering.

This work addresses the accumulation of “skill technical debt” in evolving large language model (LLM) agent skill repositories, which degrades their capabilities in retrieval, composition, and execution. To tackle this issue, the authors propose SkillOps, a novel framework that adapts the software engineering concept of technical debt to LLM skill management. SkillOps introduces typed skill contracts—categorized into Parameters, Outputs, Assumptions, Validations, and Functions (P, O, A, V, F)—and a hierarchical skill ecosystem graph to enable structured skill modeling and multidimensional health diagnostics across utility, compatibility, risk, and verifiability. Implemented as a lightweight plugin, SkillOps integrates seamlessly without modifying existing agents. Evaluated on ALFWorld, it achieves a 79.5% task success rate as a standalone agent—outperforming the strongest baseline by 8.8 percentage points—and boosts retrieval-based agents by 0.68–2.90 percentage points with negligible additional LLM invocation overhead.

Current LLM tool-use capabilities rely heavily on expensive high-end models for synthetic data generation, leading to prohibitive costs, knowledge-domain mismatch, and poor compatibility of synthetic data. Method: This paper proposes a lightweight self-improving framework—“Goal Decomposition–Self-Evolving Collaboration”—featuring (i) a novel goal decomposition mechanism for tool learning, (ii) a self-evolving training paradigm that eliminates reliance on advanced models, and (iii) integrated techniques for task decomposition, cross-architecture adaptation, and synthetic-data consistency constraints. Contribution/Results: The framework significantly improves tool-call accuracy and generalization across multi-scale and heterogeneous model architectures. It reduces high-end model invocation costs by over 90%, effectively mitigates knowledge-domain mismatch, and establishes a new pathway for autonomous, resource-efficient evolution of tool-use capabilities in lightweight LLMs.

This work addresses the limitations of existing large language model agents in tool use, which rely on agent-side optimization and thus suffer from non-shareable tool knowledge constrained by historical tasks. The authors propose ToolAtlas, a graph-based framework that, for the first time, enables persistent storage of reusable tool knowledge—including capabilities, failure boundaries, and cross-tool compositions—at the tool provider side. Agents efficiently query this knowledge through capability-guided adaptive graph traversal. ToolAtlas facilitates cross-environment and cross-framework knowledge transfer without requiring retraining or online exploration. Evaluated on two MCP benchmarks covering eight services, ToolAtlas achieves performance gains of up to 21.61% and 18.61% in pass@1 and pass@4 metrics, respectively, with relative improvements of 24.16%/16.22% and 17.49%/14.27% in cross-environment and cross-framework transfer settings.

This study addresses the unclear adaptation patterns and impacts of large language model (LLM) agent skills when reused in downstream applications. Through an empirical analysis of 1,126 adaptation instances from six prominent skill repositories, the work systematically characterizes LLM skill adaptation behaviors and constructs a taxonomy comprising 46 patterns grouped into 13 families. The research uncovers critical phenomena including a “reuse paradox,” strong cross-component dependencies, and the introduction of security-sensitive content in nearly one-fifth of adaptations. It further identifies prevalent challenges such as logic rewriting, fixing discoverability issues, and cross-tool or cross-language translation. These findings offer new empirical insights and foundational support for improving skill design, standardizing interfaces, and enabling automated adaptation of LLM-based agents.

This work addresses the inefficiency in generating and sharing new capabilities for AI agents, stemming from a lack of reusable skills during runtime. To overcome this, we propose a demand-driven, agent-centric skill production platform that introduces a novel “demand-first” paradigm for skill generation. The platform natively integrates full lifecycle skill management into Git workflows, enabling collaborative development, review, and version control among humans, scripts, and external agents within a unified state space. Leveraging mechanisms such as scoped push URLs, range-based commit ingestion, workflow state reading, and event tracing—combined with hosted repositories and registries—the system ensures auditability, recoverability, and multi-interface access (Web/REST/MCP) to skills. Empirical validation demonstrates end-to-end execution of an OS detection skill, conversion of Docker research bundles into reusable skills, and versioned submission of high-quality skill artifacts.

This study addresses the lack of systematic understanding regarding the creation, reuse, customization, and maintenance of AI agent skills as reusable software artifacts. Treating AI skills as engineered software artifacts for the first time, the authors conduct an empirical investigation based on over 40,000 skill instances drawn from public registries and GitHub repositories, employing a mixed-methods approach that combines large-scale data mining, LLM-driven SWEBOK-based knowledge classification, topic modeling, and qualitative coding. The findings reveal that 53% of reused skills remain unmodified, with reuse predominantly involving one-time copying; customization primarily serves to adapt skills to local environments; and evolution tends to follow an incremental addition pattern while preserving highly stable behavioral contracts. The study identifies six content categories of skills and six modification themes, establishing an empirical foundation for the engineering-oriented management of AI skills.

This work addresses the challenge that agents struggle to efficiently reuse successful experiences when repeatedly performing similar tasks, often resulting in redundant reasoning and excessive interaction rounds. To overcome this limitation, the paper introduces a novel framework that formalizes procedural skills as parameterized finite state machine (PFSM) subgraphs and automatically extracts, verifies, and reuses structured skills through distillation and compilation of successful execution trajectories. Evaluated on the ALFWorld and WebArena benchmarks, the proposed method significantly improves task success rates while reducing the number of required interactions, demonstrating effectiveness across language models of varying scales.

Current large language model agents often expand their skill libraries in a strictly additive manner, leading to the accumulation of redundant, outdated, or even harmful skills due to a lack of effective management. This work addresses this issue by conceptualizing the skill library as an entity requiring active refinement and introduces SkillBrew, a Pareto-aware multi-objective optimization framework. SkillBrew employs a two-stage “propose-and-verify” mechanism to dynamically balance utility, diversity, and query coverage during skill library evolution. By integrating retrieval-augmented large language models for skill evaluation and selection, the method significantly enhances skill library quality on two public benchmarks, offering a crucial step toward self-improving agents capable of maintaining efficient and adaptive skill sets.