Problem
Research questions and friction points this paper is trying to address.
Image Restoration
Continual Learning
Catastrophic Forgetting
Task Adaptation
Restoring without Forgetting: Filter-Level Continual Image Restoration via Parameter-Space Integrated Gradients
🤖 AI Summary
This study addresses the catastrophic forgetting problem encountered when adapting image inpainting models to new tasks by proposing RwF, a local adaptation framework. Leveraging integrated gradients in the parameter space, the method performs coarse-to-fine parameter attribution, revealing that task-specific knowledge is highly concentrated within a small subset of critical filters. These key filters are subsequently locally reassembled through factorized low-rank transformations, cross-task attention, and prototype-based contrastive learning. Experimental results demonstrate that RwF effectively mitigates forgetting while achieving performance comparable to full-data training approaches. Notably, it requires only one-tenth of the parameters used by LoRA, thereby realizing highly efficient and lightweight model adaptation.
📝 Abstract
Adapting image restoration models to a stream of new tasks without revisiting past data remains challenging due to catastrophic forgetting. In this work, we propose Restoring without Forgetting (RwF), a filter-level continual adaptation framework for image restoration built upon a critical observation: task-specific knowledge is centered in a small subset of filters and can be separated from those reconstructing general content. RwF first performs parameter-space integrated gradients attribution to localize degradation-critical filters in a coarse-to-fine manner. It then adapts to new tasks by generating task-specific filters from a filter bank using compact factorized low-rank transformations, further augmented with cross-task attention and prototypical contrastive learning, and lastly assembles them back only at localized positions. Experiments on six restoration tasks show that RwF effectively avoids forgetting and achieves competitive restoration quality against all-in-one methods that have full data access, and outperforms LoRA-style adaptation with $\sim$10$\times$ fewer additional parameters. Code is available at https://github.com/funkdub/Restoring-without-Forgetting.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Continual Image Restoration
Parameter-Space Integrated Gradients
Filter-Level Adaptation
Low-Rank Transformation
Catastrophic Forgetting
🔎 Similar Papers
No similar papers found.
