Towards Generalizable Deepfake Detection with Spatial-Frequency Collaborative Learning and Hierarchical Cross-Modal Fusion

2025-04-24

AI Summary

Existing deepfake detection methods suffer from poor generalization, primarily due to over-reliance on spatial-domain features while neglecting native temporal-domain artifacts and spatial–spectral interactions. To address this, we propose a spatial–frequency collaborative learning and cross-modal hierarchical fusion framework. Our approach is the first to jointly model spatial structural cues and native spectral artifacts: it introduces block-level DCT-based local spectral feature extraction, global spectral distribution modeling, and a multi-stage dynamic cross-modal fusion mechanism. Additionally, we incorporate scale-invariant differential accumulation, shallow-layer attention enhancement, and deep-layer dynamic modulation modules. Evaluated on mainstream benchmarks, our method significantly outperforms state-of-the-art approaches, achieving superior detection accuracy and cross-dataset generalization performance.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

The rapid evolution of deep generative models poses a critical challenge to deepfake detection, as detectors trained on forgery-specific artifacts often suffer significant performance degradation when encountering unseen forgeries. While existing methods predominantly rely on spatial domain analysis, frequency domain operations are primarily limited to feature-level augmentation, leaving frequency-native artifacts and spatial-frequency interactions insufficiently exploited. To address this limitation, we propose a novel detection framework that integrates multi-scale spatial-frequency analysis for universal deepfake detection. Our framework comprises three key components: (1) a local spectral feature extraction pipeline that combines block-wise discrete cosine transform with cascaded multi-scale convolutions to capture subtle spectral artifacts; (2) a global spectral feature extraction pipeline utilizing scale-invariant differential accumulation to identify holistic forgery distribution patterns; and (3) a multi-stage cross-modal fusion mechanism that incorporates shallow-layer attention enhancement and deep-layer dynamic modulation to model spatial-frequency interactions. Extensive evaluations on widely adopted benchmarks demonstrate that our method outperforms state-of-the-art deepfake detection methods in both accuracy and generalizability.

Problem

• Detect unseen deepfakes by exploiting spatial-frequency interactions
• Improve generalizability of deepfake detection across diverse forgeries
• Enhance accuracy via multi-scale spectral and cross-modal fusion

Innovation

• Multi-scale spatial-frequency analysis for deepfake detection
• Local spectral feature extraction with cascaded convolutions
• Global spectral feature extraction using differential accumulation