FSE: Continual Learning for Named Entity Recognition by Fast-Slow Experts

2026-07-24

AI Summary

This work addresses the challenge of continual learning in named entity recognition by proposing the Fast-Slow Expert (FSE) model to mitigate catastrophic forgetting and enhance cross-task knowledge sharing. The approach employs a collaborative fast-slow expert mechanism: a shared fast expert efficiently filters out irrelevant text spans, while task-specific slow experts perform classification only on the retained candidates, thereby reducing learning burden and preserving knowledge stability. Coupled with a length-decay negative sampling strategy, the model effectively balances span distribution and accelerates convergence. Experiments on OntoNotes and FewNERD demonstrate state-of-the-art performance, validating both the efficacy of individual components and the rationale behind the expert division of labor.

Abstract

Continual Learning for Named Entity Recognition (CLNER) enable models to incrementally learn new entity types without forgetting previously acquired ones. However, existing methods suffer from catastrophic forgetting and insufficient exploitation of shared information across tasks. This paper proposes FSE, a Fast-Slow Experts enhanced span-based NER model for CLNER. The shared fast expert learns token-level links to efficiently filter out unlikely spans, while the task-specific slow expert performs span classification only on the remaining candidates. It stabilizes learning by promoting knowledge sharing across tasks and maintains plasticity by reducing learning burden at each task. A length-decay negative sampling strategy to mitigate span imbalance is also introduced. Extensive experiments on OntoNotes and FewNERD synthestic datasets demonstrate that FSE achieves state-of-the-art performance in CLNER scenarios, with effectiveness of each component, empirical evidence of faster convergence and expected functionality of both experts.

Problem

• Continual Learning
• Named Entity Recognition
• Catastrophic Forgetting
• Knowledge Sharing
• Task-specific Learning

Innovation

• Continual Learning
• Named Entity Recognition
• Fast-Slow Experts
• Span-based NER
• Negative Sampling