Characterizing Photorealism and Artifacts in Diffusion Model-Generated Images

2025-02-17

AI Summary

This study investigates human ability to distinguish diffusion-model-generated images from authentic photographs and the underlying cognitive mechanisms. Method: We conducted 749,000 crowdsourced discrimination trials across state-of-the-art diffusion models released in 2024, augmented with 34,000 fine-grained human annotations. We systematically analyzed impacts of scene complexity, artifact type, stimulus presentation duration, and manual filtering on discrimination accuracy. Contribution/Results: We introduce the first taxonomy of AI-generated artifacts tailored to contemporary diffusion models, identifying core dimensions—including structural disharmony, texture anomalies, and semantic contradictions. Integrating statistical modeling with qualitative analysis, we quantify multivariate contributions and uncover a two-stage cognitive pattern: low-level visual cues dominate early-stage discrimination, while high-level semantic inconsistencies enhance late-stage accuracy. Our work establishes the first large-scale empirical framework for trustworthy AI-generated content assessment, providing both statistically grounded metrics and interpretable, human-centered discriminative criteria.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Crowd Sourcing and Human Computation
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Diffusion model-generated images can appear indistinguishable from authentic photographs, but these images often contain artifacts and implausibilities that reveal their AI-generated provenance. Given the challenge to public trust in media posed by photorealistic AI-generated images, we conducted a large-scale experiment measuring human detection accuracy on 450 diffusion-model generated images and 149 real images. Based on collecting 749,828 observations and 34,675 comments from 50,444 participants, we find that scene complexity of an image, artifact types within an image, display time of an image, and human curation of AI-generated images all play significant roles in how accurately people distinguish real from AI-generated images. Additionally, we propose a taxonomy characterizing artifacts often appearing in images generated by diffusion models. Our empirical observations and taxonomy offer nuanced insights into the capabilities and limitations of diffusion models to generate photorealistic images in 2024.

Problem

• Identifying AI-generated image artifacts
• Measuring human detection accuracy
• Proposing taxonomy for diffusion model artifacts

Innovation

• Large-scale human detection experiment
• Taxonomy of diffusion model artifacts
• Factors influencing image authenticity perception