Dynamic Feedback Engines: Layer-Wise Control for Self-Regulating Continual Learning

2025-12-25

AI Summary

Continual learning suffers from catastrophic forgetting, and existing methods often apply uniform regularization across network layers, failing to balance stability and plasticity. We observe significant entropy disparities across layers during task classification: high-entropy layers exhibit underfitting, while low-entropy layers suffer from overfitting. To address this, we propose a Layer-wise Entropy Dynamic Feedback Self-Regulation mechanism—the first to use layer-wise entropy as an online feedback signal for adaptive regularization strength adjustment per layer: reducing entropy in high-entropy layers to mitigate underfitting, and increasing entropy in low-entropy layers to alleviate overfitting, thereby guiding optimization toward wide local minima. Our approach integrates entropy-driven layer-adaptive regularization, dynamic feedback control, and wide-minima optimization, and is compatible with both replay-based and constraint-based continual learning frameworks. Extensive experiments on multiple benchmarks demonstrate substantial improvements over state-of-the-art methods, enhancing both stability and generalization in task-sequence learning.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Continual learning aims to acquire new tasks while preserving performance on previously learned ones, but most methods struggle with catastrophic forgetting. Existing approaches typically treat all layers uniformly, often trading stability for plasticity or vice versa. However, different layers naturally exhibit varying levels of uncertainty (entropy) when classifying tasks. High-entropy layers tend to underfit by failing to capture task-specific patterns, while low-entropy layers risk overfitting by becoming overly confident and specialized. To address this imbalance, we propose an entropy-aware continual learning method that employs a dynamic feedback mechanism to regulate each layer based on its entropy. Specifically, our approach reduces entropy in high-entropy layers to mitigate underfitting and increases entropy in overly confident layers to alleviate overfitting. This adaptive regulation encourages the model to converge to wider local minima, which have been shown to improve generalization. Our method is general and can be seamlessly integrated with both replay- and regularization-based approaches. Experiments on various datasets demonstrate substantial performance gains over state-of-the-art continual learning baselines.

Problem

• Addresses catastrophic forgetting in continual learning by regulating layer entropy.
• Mitigates underfitting in high-entropy layers and overfitting in low-entropy layers.
• Enhances generalization through adaptive, entropy-aware dynamic feedback mechanisms.

Innovation

• Entropy-aware dynamic feedback regulates each layer
• Reduces entropy in high-entropy layers to mitigate underfitting
• Increases entropy in low-entropy layers to alleviate overfitting