Deepfake Detection with Multi-Artifact Subspace Fine-Tuning and Selective Layer Masking

2026-01-03

arXiv.org

AI Summary

This work addresses the limited generalization of existing deepfake detection methods across datasets and real-world scenarios, primarily caused by the diverse distribution of artifacts from varying forgery techniques and the risk of semantic structure degradation when adapting to new artifacts. To tackle this, the authors propose a semantic-artifact disentanglement framework that decomposes pre-trained weights via singular value decomposition into a stable semantic principal subspace and multiple learnable artifact subspaces. Orthogonality and spectral consistency constraints are introduced to enhance subspace diversity and stability. A selective layer masking mechanism dynamically modulates fine-tuning to preserve semantic integrity. This approach is the first to explicitly model diverse forgery artifacts while maintaining semantic stability, significantly improving cross-domain generalization and mitigating overfitting to specific artifact features.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Safety and Robustness

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Deepfake detection still faces significant challenges in cross-dataset and real-world complex scenarios. The root cause lies in the high diversity of artifact distributions introduced by different forgery methods, while pretrained models tend to disrupt their original general semantic structures when adapting to new artifacts. Existing approaches usually rely on indiscriminate global parameter updates or introduce additional supervision signals, making it difficult to effectively model diverse forgery artifacts while preserving semantic stability. To address these issues, this paper proposes a deepfake detection method based on Multi-Artifact Subspaces and selective layer masks (MASM), which explicitly decouples semantic representations from artifact representations and constrains the fitting strength of artifact subspaces, thereby improving generalization robustness in cross-dataset scenarios. Specifically, MASM applies singular value decomposition to model weights, partitioning pretrained weights into a stable semantic principal subspace and multiple learnable artifact subspaces. This design enables decoupled modeling of different forgery artifact patterns while preserving the general semantic subspace. On this basis, a selective layer mask strategy is introduced to adaptively regulate the update behavior of corresponding network layers according to the learning state of each artifact subspace, suppressing overfitting to any single forgery characteristic. Furthermore, orthogonality constraints and spectral consistency constraints are imposed to jointly regularize multiple artifact subspaces, guiding them to learn complementary and diverse artifact representations while maintaining a stable overall spectral structure.

Problem

• Deepfake detection
• cross-dataset generalization
• artifact diversity
• semantic stability
• forgery artifacts

Innovation

• multi-artifact subspace
• selective layer masking
• semantic-artifact decoupling
• orthogonality constraint
• deepfake detection