An Explainable FFT-Based Spatial-Frequency Fusion Framework for Deepfake Detection

2026-07-19

AI Summary

This work addresses the challenges of media authenticity and identity fraud posed by deepfakes by proposing the MSCA-FFT framework, which, for the first time, directly leverages log-scale FFT magnitude spectra for frequency-domain feature extraction, circumventing the inverse-transform overhead inherent in conventional DCT-based approaches. The framework integrates a fine-tuned Xception-based spatial branch with a lightweight convolutional frequency branch; their features are enhanced via a Transformer encoder and efficiently fused through a multi-scale cross-attention mechanism. To enhance interpretability, Grad-CAM and LIME analyses are employed. Extensive experiments demonstrate that the model outperforms existing spatial-frequency fusion methods across multiple benchmarks, while ablation studies confirm the complementary role of the frequency branch, and visualizations consistently highlight manipulation-sensitive regions such as the eyes, mouth, nose, and facial boundaries.

Abstract

Deepfake generation has raised growing concerns regarding digital media authenticity, misinformation, identity fraud, and public trust. Recent studies show that combining spatial and frequency features leads to stronger detection results than using independently. This paper presents MSCA-FFT, a Fast Fourier Transform (FFT)-based multi-scale cross-attention framework for image-level deepfake detection. The model combines a partially fine-tuned Xception spatial branch with an FFT-based frequency branch. The frequency branch processes the log-scaled FFT magnitude spectrum through shallow convolutional layers, avoiding inverse frequency-to-image reconstruction used in DCT-based pipelines. The spatial and frequency representations are refined by transformer encoders, fused through cross-attention, and passed to an MLP classifier for real/fake prediction. Experimental results show that MSCA-FFT achieves consistently higher performance than the DCT-based state-of-the-art spatial-frequency fusion method and the compared baseline models. The ablation study further indicates that the FFT-based frequency branch provides complementary spectral cues when fused with spatial features. In addition, FFT-based frequency analysis and Grad-CAM/LIME explanations show consistent evidence around manipulation-sensitive facial regions, including the eyes, mouth, nose, and facial boundaries.

Problem

• Deepfake detection
• Spatial-frequency fusion
• Digital media authenticity
• Misinformation
• Identity fraud

Innovation

• FFT-based frequency analysis
• spatial-frequency fusion
• cross-attention mechanism
• deepfake detection
• explainable AI