Loupe: A Generalizable and Adaptive Framework for Image Forgery Detection

2025-06-20

AI Summary

To address the poor generalization, limited multi-type tampering compatibility, and model bloat in deepfake detection, this paper proposes Loupe—a lightweight, efficient joint framework unifying image-level authenticity classification and pixel-level tampering localization. Methodologically, we introduce a novel pseudo-label-guided test-time adaptation mechanism; design a patch-aware classifier jointly optimized with a conditional query-based segmentation module; and incorporate multi-scale feature fusion to significantly enhance cross-generation generalization and robustness. Evaluated on the DDL dataset, Loupe achieves state-of-the-art performance. It ranked first in the IJCAI 2025 Deepfake Detection and Localization Challenge with an overall score of 0.846, substantially outperforming existing methods in both classification accuracy and localization precision.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Security and Privacy: Data transparency and provenance

Abstract

The proliferation of generative models has raised serious concerns about visual content forgery. Existing deepfake detection methods primarily target either image-level classification or pixel-wise localization. While some achieve high accuracy, they often suffer from limited generalization across manipulation types or rely on complex architectures. In this paper, we propose Loupe, a lightweight yet effective framework for joint deepfake detection and localization. Loupe integrates a patch-aware classifier and a segmentation module with conditional queries, allowing simultaneous global authenticity classification and fine-grained mask prediction. To enhance robustness against distribution shifts of test set, Loupe introduces a pseudo-label-guided test-time adaptation mechanism by leveraging patch-level predictions to supervise the segmentation head. Extensive experiments on the DDL dataset demonstrate that Loupe achieves state-of-the-art performance, securing the first place in the IJCAI 2025 Deepfake Detection and Localization Challenge with an overall score of 0.846. Our results validate the effectiveness of the proposed patch-level fusion and conditional query design in improving both classification accuracy and spatial localization under diverse forgery patterns. The code is available at https://github.com/Kamichanw/Loupe.

Problem

• Detects and localizes diverse image forgeries effectively
• Improves generalization across manipulation types
• Enhances robustness against test distribution shifts

Innovation

• Patch-aware classifier for global classification
• Segmentation module with conditional queries
• Pseudo-label-guided test-time adaptation mechanism