Breaking the Model Forgetting Cycle in Long-Incremental 3D Object Detection

2026-07-16

AI Summary

This work addresses catastrophic forgetting in long-sequence incremental 3D object detection, where models suffer from data distribution shifts and error accumulation in pseudo-labels, leading to severe degradation in performance on previously learned categories. To mitigate this, the authors propose the LDMR framework, which introduces, for the first time, a learning dynamics signal to guide incremental learning. By monitoring dynamic changes in detection quality across categories, LDMR triggers intra-class rehearsal at appropriate stages and jointly optimizes memory buffer evolution based on scene learnability and diversity. Additionally, a periodic detection quality assessment mechanism is incorporated to break the cycle of pseudo-label error propagation. Extensive experiments on SUN RGB-D and ScanNetV2 under multiple long incremental protocols demonstrate that LDMR significantly outperforms existing methods and effectively alleviates catastrophic forgetting.

Abstract

Incremental 3D object detection requires a detector to learn novel object classes while remembering previously learned ones over sequentially arriving data. Previous methods, primarily based on pseudo-labeling, perform reasonably in short-incremental stages but still suffer from severe model forgetting when dealing with long-incremental sequences. We investigate this failure and reveal a detrimental self-reinforcing cycle: data distribution shift of novel classes causes model forgetting on old classes, which further produces accumulated error in pseudo-labeling that exacerbates model degradation. To address this issue, we draw inspiration from the human learning process and propose the \emph{Learning-Dynamics-driven Memory and Review} (LDMR) framework. LDMR monitors per-class detection quality at periodic training checkpoints and uses these learning-dynamics signals to drive two innovative mechanisms, namely (i) human-like intra-stage review that divides each incremental stage into multiple sub-stages' training and concentrates on remembering the most-forgotten objects, and (ii) scene-aware cross-stage memory evolution that evolves a memory bank to transfer knowledge between two consecutive stages by jointly considering scene learnability and diversity. Extensive experiments across multiple long-incremental protocols on indoor benchmarks SUN RGB-D and ScanNetV2 show that LDMR substantially mitigates the model forgetting and outperforms all baselines by a clear margin. Code is available at https://github.com/qianpeisheng/LDMR.

Problem

• incremental learning
• 3D object detection
• catastrophic forgetting
• long-incremental sequences
• model degradation

Innovation

• incremental learning
• 3D object detection
• model forgetting
• memory bank
• learning dynamics