Learning When to Recur: Token-Adaptive Recursion for Imbalanced Ophthalmic Domain Incremental Learning

2026-09-26

AI Summary

This study addresses catastrophic forgetting and style shift induced by class imbalance in ophthalmic incremental learning by proposing the ToRe framework. Built upon a frozen ophthalmic foundation model, ToRe employs a parameter isolation strategy to prevent knowledge overwriting. Furthermore, it introduces a novel Token Adaptive Recursion mechanism that dynamically allocates computational depth on demand to enhance minority-class feature representations, thereby achieving rehearsal-free, parameter-efficient learning. Extensive experiments across nine heterogeneous datasets demonstrate that ToRe consistently outperforms existing state-of-the-art methods while maintaining near-zero forgetting. These results effectively validate its applicability and robustness in dynamic, real-world clinical scenarios.

Abstract

Domain incremental learning is essential for adapting ophthalmic deep learning models to sequential clinical domains while preserving diagnostic expertise. Existing domain incremental learning methods predominantly address the domain shift induced by style variations. However, they often overlook the severe class imbalance inherent in real-world clinical scenarios, such as clinical referral systems. Institutions in these systems encounter drastic fluctuations in class priors, resulting in label distribution shift, a critical form of domain shift that triggers severe catastrophic forgetting. To address these challenges, we propose ToRe, a rehearsal-free and parameter-efficient framework that leverages frozen ophthalmic foundation models for robust incremental adaptation. ToRe employs a parameter isolation strategy to decouple domain-specific optimization paths, thereby helping mitigate catastrophic forgetting driven by both label distribution shift and style variations. Simultaneously, it introduces token-adaptive recursion that adaptively allocates additional computational depth across tokens, allowing simple tokens to exit the recursion loop early while subjecting complex tokens, such as those associated with lesions, to deeper recursive processing. This mechanism enhances the feature representations for minority classes, thereby supporting generalization throughout the domain incremental learning process. Extensive evaluations on nine heterogeneous datasets demonstrate that ToRe consistently outperforms state-of-the-art methods in overall performance across the three benchmarks, while maintaining near-zero forgetting. Together, these results support the applicability of ToRe to dynamic and imbalanced clinical environments. The code is available at https://github.com/Nancyolo/ToRe

Problem

• Domain Incremental Learning
• Class Imbalance
• Catastrophic Forgetting
• Label Distribution Shift
• Ophthalmic Diagnosis

Innovation

• Domain Incremental Learning
• Token-Adaptive Recursion
• Parameter Isolation
• Catastrophic Forgetting
• Class Imbalance