Beyond Retention: Orchestrating Structural Safety and Plasticity in Continual Learning for LLMs

2026-01-26

AI Summary

This work addresses the stability-plasticity trade-off in continual learning for large language models, particularly highlighting how experience replay can induce negative transfer and performance collapse in structured tasks such as code generation. The study is the first to reveal a divergence in the efficacy of experience replay between structured and unstructured tasks. To mitigate this issue, the authors propose Orthogonal Subspace Wake-up (OSW), a mechanism that identifies task-relevant parameter subspaces, retrieves critical parameters during a “wake-up” phase, and applies orthogonal gradient updates to ensure new learning does not interfere with previously acquired knowledge structures. Evaluated on a sequence of four heterogeneous tasks including code generation, OSW effectively prevents the degradation of coding capabilities caused by experience replay while maintaining strong plasticity for efficient new task acquisition, thereby achieving a synergistic balance between structural safety and adaptability.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Continual learning in Large Language Models (LLMs) faces the critical challenge of balancing stability (retaining old knowledge) and plasticity (learning new tasks). While Experience Replay (ER) is a standard countermeasure against catastrophic forgetting, its impact across diverse capabilities remains underexplored. In this work, we uncover a critical dichotomy in ER's behavior: while it induces positive backward transfer on robust, unstructured tasks (e.g., boosting performance on previous NLP classification tasks through repeated rehearsal), it causes severe negative transfer on fragile, structured domains like code generation (e.g., a significant relative drop in coding accuracy). This reveals that ER trades structural integrity for broad consolidation. To address this dilemma, we propose \textbf{Orthogonal Subspace Wake-up (OSW)}. OSW identifies essential parameter subspaces of previous tasks via a brief"wake-up"phase and enforces orthogonal updates for new tasks, providing a mathematically grounded"safety guarantee"for established knowledge structures. Empirical results across a diverse four-task sequence demonstrate that OSW uniquely succeeds in preserving fragile coding abilities where Replay fails, while simultaneously maintaining high plasticity for novel tasks. Our findings emphasize the necessity of evaluating structural safety alongside average retention in LLM continual learning.

Problem

• continual learning
• catastrophic forgetting
• structural safety
• plasticity
• large language models

Innovation

• Continual Learning
• Orthogonal Subspace Wake-up
• Structural Safety
• Experience Replay
• Large Language Models