IMFD: End-to-end Multi-Face Forgery Detection through Instruction-based Large Vision-Language Models

2026-09-17

AI Summary

本文针对多脸伪造检测问题，提出了一种基于指令的大规模视觉-语言模型的方法IMFD，通过将人脸边界框作为视觉线索整合到指令中来提高检测性能。

Abstract

The rapid increase of deepfakes has raised significant concerns due to their spread on social media. Traditional multi-face forgery detectors crop and verify each face independently, ignoring background context and inter-face relationships, which often yields suboptimal performance. To overcome these limitations, we leverage instruction-based Large Vision-Language Models (LVLMs), which can interpret entire images and follow complex textual instructions. We propose a simple yet effective single-stage multi-face forgery detector, called IMFD (Instruction-based Multi-face Forgery Detector), which is trained end-to-end to jointly localize faces and predict per-face forgery labels. Rather than treating face box prediction only as a joint objective, IMFD explicitly integrates predicted face bounding boxes into the instruction as visual cues that enhance instruction grounding and forgery detection. To support the training and evaluation of IMFD, we convert existing multi-face forgery datasets into an instruction-based format. Experimental results and analyses show that IMFD improves multi-face forgery detection by integrating face bounding boxes into the instruction, and consistently outperforms various state-of-the-art methods.

Problem

• multi-face forgery detection
• background context
• inter-face relationships

Innovation

• Instruction-based Multi-face Forgery Detector
• Large Vision-Language Models
• end-to-end training
• face bounding boxes integration
• instruction grounding