SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation

2026-06-02

AI Summary

This study addresses a novel form of visual disinformation generated by generative AI that exhibits “synthetic credibility”—characterized by realistic textual embeddings and layout design—and provides the first systematic definition and quantification of this emerging deception paradigm. The authors construct a benchmark dataset comprising 600 AI-generated images spanning six credibility types and seven dissemination styles, complemented by 450 real-image negative samples (FP450) to enable detection evaluation under controlled false-positive rates. Large-scale comparative experiments involving multimodal large language models (MLLMs), open-source detectors, commercial APIs, and human annotators reveal significant limitations: at a 5% false-positive rate, the best commercial API achieves only 57.6% true-positive rate, MLLMs average 10.5%, and humans reach 63%. These findings underscore the urgent need for detection models to move beyond superficial features toward fine-grained, robust reasoning about deep synthetic credibility cues.

Abstract

Recent generative models can now produce visual artifacts with realistic embedded text and layouts, creating a new misinformation threat: synthetic credibility. We introduce SYNCRED-Bench, a benchmark of 600 AI-generated misinformation images balanced across six credible-form categories and seven fine-grained circulation styles, together with FP450, a real-image negative set for measuring false positives. Extensive evaluation shows that existing systems remain unreliable: under a 5% false-positive-rate constraint, 15 MLLMs achieve only 10.5% true positive rate (TPR), open-source AIGC detectors achieve less than 5%, and commercial APIs reach 57.6%. Human annotators also struggled to identify synthetic credibility, reaching only 63% TPR. These findings establish synthetic credibility as a severe and underexplored visual misinformation challenge, and provide a benchmark for developing detectors that reason beyond superficial credibility cues.

Problem

• synthetic credibility
• visual misinformation
• AI-generated images
• credibility assessment
• misinformation detection

Innovation

• synthetic credibility
• visual misinformation
• AI-generated images
• benchmark
• credibility detection