SkillForge: Co-Evolving Skills and Agents via Dynamic Skill Lifecycles

2026-10-07

AI Summary

This study addresses the decision degradation in LLM agents caused by the accumulation of obsolete or harmful entries in skill libraries during training. To this end, we propose a dynamic skill lifecycle management framework. Methodologically, we design a fitness-based state transition mechanism and a pre-evaluation filtering pipeline. By integrating memory-augmented reinforcement learning, supervised fine-tuning, LLM-guided skill mutation, and selective retirement techniques, our approach enables the co-evolution of the skill library and the policy model. Experimental results demonstrate that the proposed method achieves state-of-the-art success rates across multiple benchmarks, outperforming the strongest baseline by 7.8%. Furthermore, we release SkillFurnace, an open-source dataset comprising over 5,000 annotated entries, to facilitate future research in this domain.

Abstract

Memory-augmented reinforcement learning strengthens LLM agents' ability to solve complex long-horizon tasks. Skills are one such form of memory, pairing instructions with an applicability condition over task types. However, retaining every skill indiscriminately as the policy improves lets obsolete or harmful entries accumulate and mislead the agent. We propose SkillForge, an agentic RL method that compiles and evolves the skill library through a fitness-driven skill lifecycle of trial, active, stable, and retired states, so that the skills and the model co-evolve throughout training. A pre-RL evaluation phase first uses the base model's own rollouts to pre-retire low-fitness skills, yielding a filtered library that then seeds supervised fine-tuning. Reinforcement learning takes over from this checkpoint, and at each iteration selective retirement, stabilization, and LLM-guided mutation continue to forge the skill library alongside policy optimization. Across multiple interactive agent benchmarks, SkillForge achieves the highest aggregate success rate, delivering up to 7.8% relative improvement over the strongest baseline while keeping the skill library compact throughout training. We introduce SkillFurnace, a dataset of 5k+ annotated records bundling retirement-filtered SFT trajectories, evolved skill libraries with fitness annotations, and retirement events with human-annotated failure categories to support research on skill quality and lifecycle management.

Problem

• skill library management
• reinforcement learning
• LLM agents
• memory augmentation
• long-horizon tasks

Innovation

• Skill Lifecycle Management
• Co-evolution
• Memory-augmented Reinforcement Learning
• LLM Agents
• Fitness-driven Evolution