Defense against Unauthorized Distillation in Image Restoration via Feature Space Perturbation

2025-10-09

AI Summary

Knowledge distillation attacks threaten intellectual property in image restoration models, where adversaries train student networks using teacher model outputs. Method: This paper proposes a runtime defense specifically for generative image restoration tasks. Its core innovation is the first application of Singular Value Decomposition (SVD) to anti-distillation defense: an adaptive top-k singular value amplification mechanism injects high-frequency, structured perturbations into the feature space to disrupt feature alignment during distillation, while a spatial coherence constraint preserves reconstruction fidelity. Results: Evaluated across five restoration tasks—including super-resolution and low-light enhancement—the defense reduces student model PSNR by 4 dB on average and SSIM by 60–75%, with negligible degradation (<0.1 dB PSNR loss) to teacher performance. It significantly outperforms existing defenses in both efficacy and task-agnostic robustness.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Knowledge distillation (KD) attacks pose a significant threat to deep model intellectual property by enabling adversaries to train student networks using a teacher model's outputs. While recent defenses in image classification have successfully disrupted KD by perturbing output probabilities, extending these methods to image restoration is difficult. Unlike classification, restoration is a generative task with continuous, high-dimensional outputs that depend on spatial coherence and fine details. Minor perturbations are often insufficient, as students can still learn the underlying mapping.To address this, we propose Adaptive Singular Value Perturbation (ASVP), a runtime defense tailored for image restoration models. ASVP operates on internal feature maps of the teacher using singular value decomposition (SVD). It amplifies the topk singular values to inject structured, high-frequency perturbations, disrupting the alignment needed for distillation. This hinders student learning while preserving the teacher's output quality.We evaluate ASVP across five image restoration tasks: super-resolution, low-light enhancement, underwater enhancement, dehazing, and deraining. Experiments show ASVP reduces student PSNR by up to 4 dB and SSIM by 60-75%, with negligible impact on the teacher's performance. Compared to prior methods, ASVP offers a stronger and more consistent defense.Our approach provides a practical solution to protect open-source restoration models from unauthorized knowledge distillation.

Problem

• Defending image restoration models from unauthorized knowledge distillation attacks
• Injecting structured perturbations in feature space to disrupt student learning
• Preserving teacher model performance while preventing effective distillation

Innovation

• Uses singular value decomposition on feature maps
• Amplifies top singular values for structured perturbations
• Preserves teacher output quality while hindering students