AGENTCL: Toward Rigorous Evaluation of Continual Learning in Language Agents

2026-06-01

AI Summary

Existing continual learning benchmarks struggle to systematically evaluate language agents’ ability to reuse cross-task experience under interference. This work proposes AGENTCL, a novel evaluation framework that introduces composable, controllable task streams and the MemProbe probing methodology, enabling, for the first time, fine-grained diagnosis of reliability and plasticity within memory mechanisms. Through comprehensive experiments employing non-parametric memory architectures across diverse domains—including coding, in-depth research, and linguistic reasoning—the study demonstrates that controllable task streams more clearly differentiate the performance of distinct memory designs. Furthermore, it reveals that poorly designed memory mechanisms can induce performance degradation, thereby underscoring the critical importance of robust memory architecture in continual learning for language agents.

Abstract

Language agents spend substantial inference time solving individual tasks, yet the experience acquired in one episode is often underutilized in future episodes. Continual learning expects an agent to accumulate reusable experience across a stream of tasks, improve over time, and avoid interference from irrelevant experiences. Unfortunately, existing benchmarks struggle to evaluate continual learning in language agents rigorously. Most efforts focus on retrieval and reasoning over long-context conversations or documents, while recent lifelong-adaptation benchmarks often rely on naive task streams with limited analysis of cross-task relationships, making it difficult to understand what an agent learns and reuses over time. This paper presents an evaluation framework AgentCL for continual learning in agents, centered on controlled task streams and metrics for transfer gains. AGENTCL constructs compositional streams where earlier sub-solutions, evidence, or workflows are intentionally reusable in later tasks, and contrasts them with naive streams where such reusability is not guaranteed. We use the benchmark to evaluate non-parametric memory designs for continual learning. To diagnose how memory design choices affect continual learning, we develop MemProbe, a probing method that stores interactions, insights, and skills, while filtering unreliable experiences during consolidation. Empirical analysis across coding, deep research, and language understanding/reasoning tasks shows that naive streams offer limited ability to distinguish memory designs, whereas controlled streams more clearly distinguish their plasticity. Meanwhile, naive and held-out settings often yield limited gains and can expose memory-induced degradation. These results highlight the need for stronger memory designs that balance plasticity and stable reuse.

Problem

• continual learning
• language agents
• evaluation benchmark
• task streams
• memory reuse

Innovation

• continual learning
• language agents
• evaluation framework
• compositional task streams
• memory design