Channel-wise Dynamic Knowledge Distillation via Adaptive Sample Generation for Action Recognition

2026-08-04

AI Summary

This work addresses the limitations of existing knowledge distillation methods in action recognition, which suffer from insufficient feature alignment due to fixed input samples and apply uniform distillation strength across all channels despite their varying importance. To overcome these issues, the authors propose an adaptive, sample-aware, channel-level dynamic knowledge distillation approach. It generates semantically enhanced samples by leveraging gradient and frequency-domain features, preserves critical motion information using Gaussian masking, and dynamically weights the distillation loss based on channel centroid frequencies to enable fine-grained knowledge transfer. The method achieves state-of-the-art performance across multiple video benchmarks—including UCF101, Kinetics-400, and Something-Something-v2—as well as image datasets such as CIFAR-100 and ImageNet, significantly boosting the accuracy of compact models in action recognition and demonstrating strong cross-modal generalization capabilities.

Abstract

Knowledge Distillation (KD) offers a promising yet underexplored path for compressing large action recognition models. However, existing KD methods suffer from two key limitations: 1) reliance on fixed input samples leads to suboptimal feature alignment between the frozen teacher (larger model) and the learnable student (smaller model), and 2) applying a uniform distillation strength for all channels fails to account for their varying importance in capturing distinct knowledge (e.g., motion tempo or magnitude) across training epochs. This motivates us to develop an Adaptive Sample-aware Channel-wise Dynamic (ASCD) KD approach, which operates in two stages. First, we use an adaptive sample generation module to create updated samples by incorporating semantics from sample gradients, which are derived by minimizing a feature loss weighted by channel centroid frequency differences at each layer. Meanwhile, crucial motion-related details are preserved by applying a Gaussian mask to frequency features. Second, we employ a channel-wise dynamic distillation module to train student on these generated samples, guided by sample gradients and feature frequencies. For efficiency, samples are updated periodically rather than per epoch. Extensive experiments on three video benchmarks (UCF101, Kinetics-400, Something-Something-v2) and two image datasets (CIFAR-100, ImageNet) demonstrate the state-of-the-art performance of our method. Code is available at https://github.com/mlvccn/ASCD_KD_Action.

Problem

• Knowledge Distillation
• Action Recognition
• Channel-wise Distillation
• Adaptive Sample Generation
• Model Compression

Innovation

• Channel-wise Dynamic Distillation
• Adaptive Sample Generation
• Knowledge Distillation
• Action Recognition
• Feature Frequency