catastrophic forgetting mitigation

This work addresses catastrophic forgetting in fine-tuning pretrained models, where newly acquired knowledge overwrites previously learned information. To mitigate this issue, the authors propose a function-preserving model expansion approach that mathematically duplicates and scales parameters of selected Transformer submodules during initialization. This technique enables stable training and faithful retention of original model capabilities without altering the initial functionality. By circumventing the traditional trade-off between plasticity and stability, the method achieves performance comparable to full fine-tuning while expanding only a minimal number of layers. Consequently, it fully preserves the model’s original knowledge and substantially reduces computational overhead.

This work addresses the stability-plasticity trade-off in continual learning for large language models, particularly highlighting how experience replay can induce negative transfer and performance collapse in structured tasks such as code generation. The study is the first to reveal a divergence in the efficacy of experience replay between structured and unstructured tasks. To mitigate this issue, the authors propose Orthogonal Subspace Wake-up (OSW), a mechanism that identifies task-relevant parameter subspaces, retrieves critical parameters during a “wake-up” phase, and applies orthogonal gradient updates to ensure new learning does not interfere with previously acquired knowledge structures. Evaluated on a sequence of four heterogeneous tasks including code generation, OSW effectively prevents the degradation of coding capabilities caused by experience replay while maintaining strong plasticity for efficient new task acquisition, thereby achieving a synergistic balance between structural safety and adaptability.

Continual learning (CL) of pretrained models faces a fundamental trade-off between stability and plasticity, especially under significant distribution shifts across domains. To address this, we propose a “adapt-then-learn” paradigm: before processing each new task, a lightweight, prototype-guided adaptation is applied to the backbone network, followed by standard CL training. This adaptation aligns embedding spaces via prototype-aware optimization, theoretically unifying class-prototype consistency and feature discriminability—thereby overcoming the performance bottleneck inherent in frozen or full-parameter fine-tuning. The adaptation module is plug-and-play, seamlessly integrating with mainstream CL frameworks such as Prompt Tuning. Extensive experiments on multiple benchmarks demonstrate that our method significantly mitigates catastrophic forgetting, enhances cross-domain transfer capability, and improves generalization across tasks.

Existing replay-based continual learning methods suffer from poor scalability and high computational overhead, while failing to effectively integrate parameter-efficient fine-tuning techniques—such as Low-Rank Adaptation (LoRA) and model merging—into multi-task adaptation. To address this, we propose a novel replay framework that deeply couples staged knowledge consolidation with sequential model merging tailored for task-ordered streams. It is the first to systematically integrate LoRA, task arithmetic, and collaborative optimization within a unified replay paradigm. Our approach reduces required replay sample volume by up to 55%, significantly improving training efficiency and model scalability. Empirically, it outperforms individual baseline methods across diverse multi-task sequences, achieving organic synergy between replay mechanisms and parameter-efficient adaptation. This work establishes a new paradigm for lightweight, accumulative continual learning models.

Addressing the stability-plasticity dilemma, severe catastrophic forgetting, linear adapter expansion, and limited knowledge reuse in continual learning with pre-trained models (PTMs), this paper proposes the Self-Expanding Modular Adapter framework (SEMA). Methodologically, SEMA introduces: (1) a distribution-shift-driven self-expansion mechanism that dynamically detects task transitions via multi-level representation descriptors and incrementally adds or removes adapters on demand; (2) an expandable weighted routing scheme enabling adapter mixture outputs and sublinear parameter growth; and (3) a frozen-backbone, lightweight fine-tuning paradigm. Under the challenging replay-free setting, SEMA achieves state-of-the-art performance, significantly outperforming existing PTM-based continual learning approaches. Empirical results demonstrate that SEMA simultaneously attains sublinear adapter scaling and enhanced generalization—validating the synergistic benefits of controlled architectural expansion and adaptive knowledge integration.

This work addresses a critical gap in continual learning research: while most existing methods focus on mitigating catastrophic forgetting, they largely overlook the conditions under which forward transfer—where knowledge from past tasks benefits new ones—can be effectively realized. The paper introduces, for the first time, a systematic three-condition framework that characterizes when forward transfer is feasible and proposes Transfer-Selective Replay (TSR), a novel method that leverages a zero-training-overhead task signature mechanism to automatically identify and replay only those historical samples beneficial to the current task. TSR integrates knowledge distillation to preserve performance on previous tasks while explicitly promoting forward transfer as a first-class objective. Experiments demonstrate that TSR significantly enhances forward transfer across both homogeneous and heterogeneous task sequences and consistently outperforms existing replay-based baselines, especially under limited replay budgets.

This work addresses the critical challenge in continual learning of mitigating catastrophic forgetting during downstream fine-tuning while preserving capabilities acquired during upstream training. The authors propose treating “robustness to subsequent fine-tuning” as a first-class objective in upstream training and systematically investigate data scheduling strategies across a three-stage pipeline—pretraining, post-training, and downstream fine-tuning. Their key finding is that early exposure to post-training data during pretraining—termed “early data exposure”—consistently outperforms pure post-training or conventional mixing strategies, yielding superior trade-offs between upstream knowledge retention and downstream task performance across model scales from 135M to 1B parameters. This approach complements regularization techniques such as replay and Dropout and, under fixed compute budgets, reveals an optimal data allocation scheme.

This work addresses the limitation of traditional continual learning, which overly emphasizes preserving old knowledge to approximate joint training while neglecting real-time adaptation to new tasks in non-stationary environments. The problem is reformulated as an online optimization framework, with average lifelong error as the performance metric, and a notion of transfer efficiency is introduced to characterize the trade-off between stability and transient error induced by historical knowledge. Drawing on critical task duration theory, the study identifies conditions under which past knowledge shifts from beneficial to detrimental. Building on this insight, the paper proposes a predictive continual learning paradigm that dynamically models future tasks to optimize expected performance. Integrating online learning, transfer efficiency analysis, and convergence theory, an algorithm based on task-window interpolation is developed and validated on image classification and reinforcement learning benchmarks, demonstrating significant superiority over both joint training and independent learning, especially under distribution shift.

This work addresses the challenge of catastrophic forgetting in artificial neural networks during continual learning. Inspired by biological sleep mechanisms, the authors propose an unsupervised, sleep-like replay phase that consolidates all previously acquired knowledge in a unified manner after sequential task learning, without requiring immediate intervention during training. Departing from conventional continual learning paradigms, this approach demonstrates for the first time that a single replay session conducted after the entire task sequence can effectively mitigate forgetting. The study further reveals that information from prior tasks undergoes gradual decay rather than abrupt overwriting during new learning. Experimental results show that this strategy substantially recovers performance across all previously learned tasks, offering a novel and biologically inspired perspective on memory consolidation in continual learning systems.

This study addresses the dual challenges of emerging domains and data drift faced by large language models in dynamic environments. The authors decompose continual learning into spatial (new domains) and temporal (data drift) dimensions and propose the first mechanism-agnostic, unified evaluation protocol. Within this consistent framework, they systematically compare the adaptability of diverse approaches—including prompt engineering (e.g., GEPA, ACE), supervised fine-tuning (SFT, SDFT), online reinforcement learning (GRPO, SDPO), and context compression (Cartridges, In-place TTT). Their analysis reveals that effective adaptation depends on aligning update mechanisms with specific environmental dynamics: online reinforcement learning excels at knowledge updating yet is sensitive to noise; distillation-based methods offer stability but struggle to correct outdated facts; and prompt-based strategies respond rapidly but exhibit limited generalization.