SegACIL: Solving the Stability-Plasticity Dilemma in Class-Incremental Semantic Segmentation

2024-12-14

arXiv.org

AI Summary

To address catastrophic forgetting in continual semantic segmentation—particularly severe at pixel-/point-level granularity in 2D image and 3D point cloud settings due to reliance on multiple SGD rounds—this paper proposes the first single-pass, linear closed-form continual learning framework. Our method enforces strict theoretical trade-offs between stability (for old classes) and plasticity (for new classes) via parameter-space projection constraints, incrementally decoupled weight updates, and a replay-free lightweight architecture. Crucially, we provide the first formal proof in segmentation that its asymptotic performance matches joint training. Evaluated on Pascal VOC2012 under sequential, disjoint, and overlapping incremental settings, our approach achieves average mIoU gains of 2.1–3.8% over state-of-the-art methods while reducing training overhead by over 90%. This establishes a new benchmark for efficiency and accuracy in continual segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Life-Long and Continual Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance

Abstract

While deep learning has made remarkable progress in recent years, models continue to struggle with catastrophic forgetting when processing continuously incoming data. This issue is particularly critical in continual learning, where the balance between retaining prior knowledge and adapting to new information-known as the stability-plasticity dilemma-remains a significant challenge. In this paper, we propose SegACIL, a novel continual learning method for semantic segmentation based on a linear closed-form solution. Unlike traditional methods that require multiple epochs for training, SegACIL only requires a single epoch, significantly reducing computational costs. Furthermore, we provide a theoretical analysis demonstrating that SegACIL achieves performance on par with joint learning, effectively retaining knowledge from previous data which makes it to keep both stability and plasticity at the same time. Extensive experiments on the Pascal VOC2012 dataset show that SegACIL achieves superior performance in the sequential, disjoint, and overlap settings, offering a robust solution to the challenges of class-incremental semantic segmentation. Code is available at https://github.com/qwrawq/SegACIL.

Problem

• Class-incremental semantic segmentation without catastrophic forgetting
• Eliminating iterative gradient-based optimization for computational efficiency
• Exemplar-free approach for privacy-sensitive multimedia environments

Innovation

• Closed-form solution for CSS
• Exemplar-free continual learning
• Single-pass new sample processing