Semantic Projection for Continual Self-Evolution of Language Agents

2026-09-28

AI Summary

This study addresses the problem of catastrophic forgetting during the continuous skill evolution of language agents by proposing a semantic scope projection evolution method. This approach transfers the principle of gradient projection into behavioral space, achieving for the first time parameter-gradient-free semantic projection of natural language skills, thereby enabling shared skills to be stably revised without exposing domain identities. Furthermore, it introduces a behavioral space projection based on orthogonal gradient descent alongside a skill compatibility evaluation algorithm tailored for non-stationary task streams. Experimental results demonstrate that the proposed method significantly enhances final cross-domain capabilities and effectively mitigates forgetting, maintaining optimal average performance even when the execution model is replaced.

Abstract

Language-model agents increasingly rely on persistent natural-language skills to adapt beyond their frozen model parameters. When a shared skill is repeatedly revised from a non-stationary, heterogeneous task stream, however, improvements for new tasks can overwrite procedures needed for earlier ones. In continual learning, Orthogonal Gradient Descent (OGD) addresses analogous interference by projecting a new-task gradient onto a subspace that locally preserves prior predictions. Natural-language skill revisions, however, have neither gradients nor a canonical vector space in which such a projection can be performed. We introduce \emph{Semantic-Scope Projected Evolution} (SSPE), which transfers the functional principle of gradient projection from parameter space to behavior space. SSPE treats an unconstrained skill revision as a proposed update, identifies acquired capabilities with which it may interfere, and uses the observed gains and regressions to construct a compatible revision rather than merely rejecting the update. This enables one shared skill to evolve across latent and recurring task contexts without exposing semantic domain identities to the evolution model. Across controlled synthetic streams and heterogeneous real-agent benchmarks, SSPE improves final cross-domain competence and mitigates forgetting relative to strong skill-evolution baselines. The evolved skill also retains the strongest average performance after transfer to a different executor model. These results establish semantic projection as a promising principle for stable and adaptive self evolution of language agents.

Problem

• continual learning
• language agents
• catastrophic forgetting
• skill evolution
• semantic projection

Innovation

• Semantic Projection
• Continual Learning
• Language Agents
• Skill Evolution
• Catastrophic Forgetting