Scalable Strategies for Continual Learning with Replay

2025-05-18

AI Summary

Existing replay-based continual learning methods suffer from poor scalability and high computational overhead, while failing to effectively integrate parameter-efficient fine-tuning techniques—such as Low-Rank Adaptation (LoRA) and model merging—into multi-task adaptation. To address this, we propose a novel replay framework that deeply couples staged knowledge consolidation with sequential model merging tailored for task-ordered streams. It is the first to systematically integrate LoRA, task arithmetic, and collaborative optimization within a unified replay paradigm. Our approach reduces required replay sample volume by up to 55%, significantly improving training efficiency and model scalability. Empirically, it outperforms individual baseline methods across diverse multi-task sequences, achieving organic synergy between replay mechanisms and parameter-efficient adaptation. This work establishes a new paradigm for lightweight, accumulative continual learning models.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Future deep learning models will be distinguished by systems that perpetually learn through interaction, imagination, and cooperation, blurring the line between training and inference. This makes continual learning a critical challenge, as methods that efficiently maximize bidirectional transfer across learning trajectories will be essential. Replay is on track to play a foundational role in continual learning, allowing models to directly reconcile new information with past knowledge. In practice, however, replay is quite unscalable, doubling the cost of continual learning when applied naively. Moreover, the continual learning literature has not fully synchronized with the multi-task fine-tuning literature, having not fully integrated highly scalable techniques like model merging and low rank adaptation into a replay-enabled toolset that can produce a unified model in the face of many sequential tasks. In this paper, we begin by applying and analyzing low rank adaptation in a continual learning setting. Next, we introduce consolidation, a phasic approach to replay which leads to up to 55% less replay samples being needed for a given performance target. Then, we propose sequential merging, an offshoot of task arithmetic which is tailored to the continual learning setting and is shown to work well in combination with replay. Finally, we demonstrate that the developed strategies can operate synergistically, resulting in a highly scalable toolset that outperforms standalone variants.

Problem

• Addressing scalability issues in continual learning with replay
• Integrating multi-task fine-tuning techniques into continual learning
• Reducing replay samples needed for performance targets

Innovation

• Low rank adaptation for continual learning
• Phasic replay reducing needed samples
• Sequential merging tailored for continual learning