🤖 AI Summary
Continual learning on edge devices faces significant challenges due to excessive memory overhead and deployment difficulty. To address this, we propose a low-memory, compact continual learning framework. First, we introduce a novel dual-dimensional generalization metric—quantifying both *learning plasticity* and *memory stability*—to dynamically freeze layers exhibiting high generalization capability. Second, we design a lightweight feature-stability regularization that jointly preserves model generalizability and retains historical knowledge. Third, we integrate layer-wise adaptive freezing with edge-specific deployment optimizations (e.g., quantization-aware pruning and runtime memory scheduling). Evaluated across multiple benchmarks, our method reduces memory footprint by up to 6.16× compared to state-of-the-art approaches, while maintaining or improving accuracy. This substantially enhances the practicality and efficiency of continual learning on resource-constrained edge platforms.
📝 Abstract
Continual learning (CL) is a technique that enables neural networks to constantly adapt to their dynamic surroundings. Despite being overlooked for a long time, this technology can considerably address the customized needs of users in edge devices. Actually, most CL methods require huge resource consumption by the training behavior to acquire generalizability among all tasks for delaying forgetting regardless of edge scenarios. Therefore, this paper proposes a compact algorithm called LightCL, which evaluates and compresses the redundancy of already generalized components in structures of the neural network. Specifically, we consider two factors of generalizability, learning plasticity and memory stability, and design metrics of both to quantitatively assess generalizability of neural networks during CL. This evaluation shows that generalizability of different layers in a neural network exhibits a significant variation. Thus, we $ extit{Maintain Generalizability}$ by freezing generalized parts without the resource-intensive training process and $ extit{Memorize Feature Patterns}$ by stabilizing feature extracting of previous tasks to enhance generalizability for less-generalized parts with a little extra memory, which is far less than the reduction by freezing. Experiments illustrate that LightCL outperforms other state-of-the-art methods and reduces at most $ extbf{6.16$ imes$}$ memory footprint. We also verify the effectiveness of LightCL on the edge device.