Noradrenergic-inspired gain modulation attenuates the stability gap in joint training

2025-07-18

AI Summary

In continual learning, models exhibit transient performance degradation on previously learned tasks upon task switching—a phenomenon termed the “stability gap”—which persists even under ideal joint-training conditions, revealing insufficient robustness of existing methods against catastrophic forgetting. To address this, we draw inspiration from the locus coeruleus–norepinephrine system’s neuromodulatory mechanism and propose, for the first time, an uncertainty-aware dynamic gain modulation method for artificial neural networks, establishing a biologically grounded dual-timescale optimization framework. Evaluated on MNIST and CIFAR benchmarks under domain-incremental and class-incremental settings, our approach significantly narrows the stability gap and enhances model robustness. Moreover, it uncovers a neurocomputational analogy: gain modulation facilitates stable knowledge integration, offering an interpretable, biologically inspired paradigm for continual learning.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Economics, Online Markets and Human Computation: Social networks and social learning
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent studies in continual learning have identified a transient drop in performance on mastered tasks when assimilating new ones, known as the stability gap. Such dynamics contradict the objectives of continual learning, revealing a lack of robustness in mitigating forgetting, and notably, persisting even under an ideal joint-loss regime. Examining this gap within this idealized joint training context is critical to isolate it from other sources of forgetting. We argue that it reflects an imbalance between rapid adaptation and robust retention at task boundaries, underscoring the need to investigate mechanisms that reconcile plasticity and stability within continual learning frameworks. Biological brains navigate a similar dilemma by operating concurrently on multiple timescales, leveraging neuromodulatory signals to modulate synaptic plasticity. However, artificial networks lack native multitimescale dynamics, and although optimizers like momentum-SGD and Adam introduce implicit timescale regularization, they still exhibit stability gaps. Inspired by locus coeruleus mediated noradrenergic bursts, which transiently enhance neuronal gain under uncertainty to facilitate sensory assimilation, we propose uncertainty-modulated gain dynamics - an adaptive mechanism that approximates a two-timescale optimizer and dynamically balances integration of knowledge with minimal interference on previously consolidated information. We evaluate our mechanism on domain-incremental and class-incremental variants of the MNIST and CIFAR benchmarks under joint training, demonstrating that uncertainty-modulated gain dynamics effectively attenuate the stability gap. Finally, our analysis elucidates how gain modulation replicates noradrenergic functions in cortical circuits, offering mechanistic insights into reducing stability gaps and enhance performance in continual learning tasks.

Problem

• Addresses stability gap in continual learning tasks
• Balances rapid adaptation and robust retention
• Proposes gain modulation to mitigate forgetting

Innovation

• Noradrenergic-inspired gain modulation balances plasticity
• Uncertainty-modulated gain dynamics attenuates stability gap
• Two-timescale optimizer integrates knowledge with minimal interference