Difference Feature Map Distillation: Transferring Inter-Sample Relational Knowledge Towards Efficient Transformer-Based Tracking

2026-06-01

2026 IEEE Intelligent Vehicles Symposium (IV)

AI Summary

This study addresses the substantial computational overhead of Transformer-based trackers, which hinders real-time, low-power deployment in autonomous driving. To this end, we propose a relational knowledge distillation framework based on differential feature mapping (DFM-KD). Unlike conventional pointwise error minimization approaches, DFM-KD explicitly aligns inter-sample feature discrepancies, redirecting knowledge transfer toward relational dynamics of appearance variations rather than absolute activation similarities. Experimental results demonstrate that DFM-KD significantly enhances the robustness of lightweight models, achieving superior tracking precision and success rates compared to existing feature-level distillation methods. This work establishes a novel paradigm for efficient visual tracking.

Abstract

In autonomous driving perception, visual object tracking systems must satisfy stringent latency and power constraints while remaining robust in complex and dynamic environments. Although transformer-based trackers achieve state-of-the-art accuracy, their substantial computational and memory overheads hinder deployment on real-time, resource-constrained platforms. To move toward this goal, we propose Difference Feature Map Knowledge Distillation (DFM-KD), a novel relational distillation framework tailored for transformer-based visual object tracking. Unlike conventional feature distillation methods that minimize point-wise discrepancies (e.g., mean squared error) between teacher and student feature representations, DFM-KD transfers knowledge through inter-sample feature differences, explicitly aligning the relational structure of the feature space. By distilling how the teacher models appearance variation and consistency across samples, rather than enforcing similarity in absolute activations, DFM-KD enables the student to better capture the structural dynamics of visual changes within a batch. As a result, the distilled model exhibits enhanced feature robustness and improved tracking performance. Extensive experiments demonstrate that DFM-KD consistently outperforms conventional feature-level distillation methods in both tracking precision and success rates.

Problem

• visual object tracking
• transformer
• knowledge distillation
• computational overhead
• autonomous driving

Innovation

• Knowledge Distillation
• Difference Feature Map
• Relational Distillation
• Visual Object Tracking
• Transformer