Rep2Skill: Representation-Guided Skill Self-Evolution for LLM Agents

2026-09-30

AI Summary

This study addresses the limitation that existing skill evolution in LLM-based agents is confined to the textual space, overlooking rich execution state information embedded within internal representations. To overcome this, we propose Rep2Skill, a framework that, for the first time, incorporates internal representations into the closed loop of external skill self-evolution. By modeling representation trajectories to localize anomalous steps and translating cross-modal signals into actionable textual feedback for skill refinement, our approach transcends the constraints of purely text-based reflection. Experimental results demonstrate that Rep2Skill significantly outperforms text-only baselines across two environments and multiple open-source models. This work establishes a novel paradigm for agent self-optimization that eliminates reliance on external, more capable models for auxiliary guidance.

Abstract

Textual skills enable large language model (LLM) based agents to accumulate reusable procedural knowledge without updating model parameters. Yet existing skill evolution remains largely confined to the text space: an optimizer must diagnose success and failure patterns, and revise skills solely from long execution trajectories and sparse task outcomes. This text-only paradigm leaves the agent's internal representations, which contain rich records of its evolving execution state, outside the skill optimization loop. We ask whether an agent can improve its external textual skills by reflecting on its own internal representations. We introduce Rep2Skill, a representation-guided framework for self-evolution on agent skills. Specifically, upon the collected agent rollouts, Rep2Skill models their internal model representation trajectories to localize turns that deviate from successful execution dynamics, and it further interprets these signals alongside the execution contexts as actionable textual feedback for targeted skill revision. Experiments on two agent environments with two open-source LLMs show that Rep2Skill consistently outperforms text-only approaches in the self-evolution setting, where the same LLM serves as both executor and optimizer without a stronger external model. This establishes a promising direction moving agent self-improvement beyond text-only reflection.

Problem

• LLM agents
• skill self-evolution
• internal representations
• text-only optimization
• self-improvement

Innovation

• Representation-Guided
• Skill Self-Evolution
• LLM Agents
• Internal Representations
• Self-Improvement