ACLArena: Agent Continue Learning in Multi-stage Post-training

2026-09-20

AI Summary

本文针对多阶段训练后代理持续学习问题，提出ACLArena框架，通过分析遗忘和泛化机制，比较多种方法，并提出结合高质量轨迹离线回放与多专家LoRA网络的新方案。

Abstract

Building general-purpose agents for industrial deployment requires integrating multiple capabilities, each typically acquired at a distinct stage of training. Yet there is currently no well-established recipe for Agent Continual Learning (ACL), with little understanding of the trade-offs among existing integration paradigms. To address this gap, we introduce ACLArena, a framework for comprehensively studying, analyzing, and evaluating ACL. We first build a sequential training pipeline and conduct an in-depth analysis that explains the mechanisms of forgetting and generalization from two complementary perspectives, the model level and the token level. Guided by these analyses, we systematically compare multi-teacher on-policy distillation, self-distilled fine-tuning, and model merging to assess their ability to recover previously learned capabilities while preserving newly acquired ones. Through extensive experiments, we develop a detailed understanding of how capabilities transfer across stages. Finally, we propose a new ACL recipe that combines offline replay over high-quality trajectories with a routed network of multiple LoRA experts each specialized via RL, substantially improving the agent's ability to learn across multiple domains. Comprehensive experiments on four reasoning and agentic tasks, evaluated under both in-domain and out-of-domain settings, demonstrate the value of our analysis and the effectiveness of our approach.

Problem

• Agent Continual Learning
• multi-stage post-training
• capability integration

Innovation

• Agent Continual Learning
• Sequential Training Pipeline
• Offline Replay
• LoRA Experts
• Multi-domain Learning