Distilled transformers with locally enhanced global representations for face forgery detection

2024-12-01

Pattern Recognition

AI Summary

Existing deepfake detection models struggle to simultaneously capture local details (via CNNs) and model global semantics (via Transformers), suffer from attention instability in hybrid architectures, and exhibit poor generalization under limited training data. To address these issues, this paper proposes a lightweight distilled Transformer architecture. Its core innovation is a local-enhanced global representation distillation paradigm: leveraging hierarchical attention mechanisms and local window-based feature enhancement, the framework jointly compresses model capacity while preserving discriminative forgery cues within a knowledge distillation framework. Evaluated on FaceForensics++ and Celeb-DF, the method achieves state-of-the-art performance, with a maximum AUC of 99.2%. It reduces parameter count by 38% and accelerates inference by 2.1× compared to baseline models, demonstrating superior efficiency and generalization—especially in low-data regimes.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Problem

• Facial Forgery Detection
• CNN-Transformer Integration
• Limited Labeled Data

Innovation

• DTN (Distillation Transformer Network)
• Multi-Attention Scaling Module
• Deep Fake Self-Distillation