A Benchmark Dataset for MLLM-Generated Image Detection: GPT Image2 & Nano Banana2

2026-08-02

AI Summary

Existing benchmarks for detecting AI-generated images struggle to handle the high-fidelity, multimodal outputs produced by Multimodal Large Language Models (MLLMs). To address this gap, this work introduces the first MLLM-oriented detection benchmark dataset, encompassing real-world scenarios and featuring three generation protocols: direct generation, reference-based reconstruction, and localized editing. The benchmark systematically evaluates the performance degradation of current detectors under these conditions. Building upon these insights, the authors propose a Structure Artifact Prior-guided Dual-Stream Prompting framework (SAP-DSP), which enhances discrimination of MLLM-generated images through dual-stream prompt learning and structure-aware routing fusion. Experiments demonstrate that the proposed benchmark effectively exposes the failure of existing methods in capturing structural details and rendering text, while SAP-DSP achieves consistently robust detection performance across diverse sample types.

Abstract

The realism of images generated by multimodal large language models (MLLMs), such as GPT Image2 and Nano Banana2, has improved rapidly in recent years. Compared with early generative models, current models have made clear progress in text rendering. They can produce high-quality images that closely resemble real-world application scenarios. The enhanced generation capabilities of current MLLMs pose increasingly severe challenges to AI-generated image detection. Detection is no longer limited to identifying obvious artifacts left by early generators. Instead, it requires systematic and realistic benchmarks for the new generation of generated content. However, most existing benchmarks are still built around early generative models and cannot fully evaluate the forensic challenges introduced by high-quality and multi-form generated images. To address this gap, this paper constructs a benchmark dataset for detecting images generated by MLLMs. The benchmark covers several realistic application scenarios and adopts three generation protocols to simulate direct generation, reference-based reconstruction, and local editing. Based on this benchmark, we evaluate detector degradation from traditional scenarios to MLLM-generated images and analyze false positive rates and false negative rates across three sample types, revealing the failure modes of existing methods. We further propose a structural-artifact-prior-guided dual-stream prompt framework (SAP-DSP) as a strong baseline. SAP-DSP uses dual-stream prompt learning and structure-aware routing fusion to improve representation learning. Extensive experiments show that the proposed benchmark exposes the performance degradation of existing detectors on high-quality generated images, while SAP-DSP achieves more stable detection results on this benchmark. Our code and dataset are publicly available at https://github.com/xbrainnet/SAP-DSP.

Problem

• MLLM-generated image detection
• benchmark dataset
• AI-generated image forensics
• multimodal large language models
• image authenticity

Innovation

• MLLM-generated image detection
• benchmark dataset
• structural artifact prior
• dual-stream prompt learning
• structure-aware fusion