LibContinual: A Comprehensive Library towards Realistic Continual Learning

2025-12-26

AI Summary

Continual learning (CL) suffers from catastrophic forgetting and suffers from inconsistent evaluation protocols and fragmented implementations, hindering reproducibility. To address these issues, this work introduces the first unified, real-world-oriented CL benchmark library. It systematically identifies and challenges three prevalent implicit assumptions in mainstream CL: offline access to past data, unconstrained memory budgets, and intra-class semantic homogeneity. The benchmark establishes a new paradigm featuring online-style evaluation, unified memory budget allocation, and class randomization. Its modular architecture integrates 19 representative CL algorithms and supports strict online training, dynamic memory management, and cross-task semantic perturbations. Empirical results demonstrate significant performance degradation of most state-of-the-art methods under realistic constraints. The open-source, extensible platform fosters research on resource-aware and semantically robust continual learning.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Computer Vision: Adversarial Attacks & Robustness
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

A fundamental challenge in Continual Learning (CL) is catastrophic forgetting, where adapting to new tasks degrades the performance on previous ones. While the field has evolved with diverse methods, this rapid surge in diverse methodologies has culminated in a fragmented research landscape. The lack of a unified framework, including inconsistent implementations, conflicting dependencies, and varying evaluation protocols, makes fair comparison and reproducible research increasingly difficult. To address this challenge, we propose LibContinual, a comprehensive and reproducible library designed to serve as a foundational platform for realistic CL. Built upon a high-cohesion, low-coupling modular architecture, LibContinual integrates 19 representative algorithms across five major methodological categories, providing a standardized execution environment. Meanwhile, leveraging this unified framework, we systematically identify and investigate three implicit assumptions prevalent in mainstream evaluation: (1) offline data accessibility, (2) unregulated memory resources, and (3) intra-task semantic homogeneity. We argue that these assumptions often overestimate the real-world applicability of CL methods. Through our comprehensive analysis using strict online CL settings, a novel unified memory budget protocol, and a proposed category-randomized setting, we reveal significant performance drops in many representative CL methods when subjected to these real-world constraints. Our study underscores the necessity of resource-aware and semantically robust CL strategies, and offers LibContinual as a foundational toolkit for future research in realistic continual learning. The source code is available from href{https://github.com/RL-VIG/LibContinual}{https://github.com/RL-VIG/LibContinual}.

Problem

• Addresses fragmented research and reproducibility issues in continual learning
• Identifies and challenges unrealistic assumptions in mainstream continual learning evaluations
• Proposes a unified library to assess methods under real-world constraints

Innovation

• A modular library integrating 19 algorithms for standardized continual learning
• Introduces strict online settings and memory budget for realistic evaluation
• Proposes category-randomized setting to test semantic robustness in learning