Collaboratively Guided Adversarial Robust Distillation with Teacher-Favorable Examples

2026-10-08

AI Summary

This study addresses the limitation of existing adversarial distillation methods, where insufficient exploitation of beneficial teacher samples constrains the efficiency of robust knowledge transfer. To this end, we propose CGARD, a framework that introduces constrained teacher-collaborative samples to jointly optimize student adversarial training and teacher-collaborative guidance, thereby deeply integrating adversarial supervision with collaborative instruction. The proposed method synergistically combines adversarial distillation, cross-entropy constrained optimization, and collaborative sample generation techniques. Experimental evaluations on CIFAR datasets demonstrate that CGARD significantly enhances both white-box and black-box adversarial robustness, outperforming mainstream baseline methods.

Abstract

Adversarial distillation transfers robustness from high-capacity teachers to compact students. Existing adversarial distillation methods mainly use teacher predictions on clean or adversarial examples to supervise student learning. However, teacher-favorable supervision within the perturbation neighborhood remains underexplored in adversarial distillation. We therefore propose Collaboratively Guided Adversarial Robust Distillation (CGARD), which jointly optimizes distinct student-adversarial and teacher-collaborative examples within the same perturbation neighborhood. The teacher-collaborative example is constrained to incur no greater cross-entropy loss under the teacher than the clean input. CGARD combines collaborative teacher guidance with adversarial teacher supervision to improve robust knowledge transfer. Experiments on CIFAR-10 and CIFAR-100, including white-box evaluation and additional black-box transfer evaluation, demonstrate consistent robustness improvements over strong adversarial distillation baselines.

Problem

• Adversarial Distillation
• Robustness Transfer
• Perturbation Neighborhood
• Teacher Supervision

Innovation

• Adversarial Distillation
• Robust Knowledge Transfer
• Teacher-Favorable Examples
• Collaborative Guidance
• Perturbation Neighborhood