On the Holistic Approach for Detecting Human Image Forgery

2026-01-08

arXiv.org

AI Summary

Existing methods for human image forgery detection are often limited to either facial or full-body scenarios, exhibiting constrained generalization capabilities. This work proposes HuForDet, a unified detection framework that introduces a novel dual-branch architecture integrating both facial and full-body contextual information. The framework synergistically combines heterogeneous experts operating in RGB and frequency domains, an adaptive Laplacian of Gaussian (LoG) operator, a multimodal large language model, and a dynamic confidence-weighted fusion mechanism to effectively identify diverse human-centric forgeries. Evaluated on the newly curated HuFor dataset, HuForDet significantly outperforms current state-of-the-art approaches, achieving leading detection accuracy and robustness across a wide range of forgery types.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

The rapid advancement of AI-generated content (AIGC) has escalated the threat of deepfakes, from facial manipulations to the synthesis of entire photorealistic human bodies. However, existing detection methods remain fragmented, specializing either in facial-region forgeries or full-body synthetic images, and consequently fail to generalize across the full spectrum of human image manipulations. We introduce HuForDet, a holistic framework for human image forgery detection, which features a dual-branch architecture comprising: (1) a face forgery detection branch that employs heterogeneous experts operating in both RGB and frequency domains, including an adaptive Laplacian-of-Gaussian (LoG) module designed to capture artifacts ranging from fine-grained blending boundaries to coarse-scale texture irregularities; and (2) a contextualized forgery detection branch that leverages a Multi-Modal Large Language Model (MLLM) to analyze full-body semantic consistency, enhanced with a confidence estimation mechanism that dynamically weights its contribution during feature fusion. We curate a human image forgery (HuFor) dataset that unifies existing face forgery data with a new corpus of full-body synthetic humans. Extensive experiments show that our HuForDet achieves state-of-the-art forgery detection performance and superior robustness across diverse human image forgeries.

Problem

• deepfakes
• human image forgery
• face forgery
• full-body synthesis
• AIGC

Innovation

• holistic forgery detection
• dual-branch architecture
• adaptive Laplacian-of-Gaussian
• Multi-Modal Large Language Model (MLLM)
• human image forgery dataset