Playing the Imitation Game: How Perceived Generated Content Shapes Player Experience

2026-02-15

AI Summary

This study investigates players’ perceptual differences between AI-generated and human-designed game content and how these perceptions influence gameplay experience. Employing a mixed-methods approach—including user experiments, questionnaires, and interviews—the research compares procedurally generated and handcrafted levels in *Super Mario Bros.* and *Sokoban*. Findings reveal that participants were unable to reliably identify the true origin of the content; however, their beliefs about the creator’s identity significantly shaped their evaluations. Levels mistakenly believed to be human-designed were rated as more enjoyable and aesthetically pleasing, whereas those perceived as AI-generated were judged as more difficult and frustrating. The results indicate a spontaneous negative bias toward AI-generated content, demonstrating that subjective beliefs—not actual provenance—primarily drive experiential judgments, thereby highlighting the critical role of cognitive biases in human-AI collaborative content creation.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Multiagent Systems: Adversarial Agents

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

With the fast progress of generative AI in recent years, more games are integrating generated content, raising questions regarding how players perceive and respond to this content. To investigate, we ran a mixed-method survey on the games Super Mario Bros. and Sokoban, comparing procedurally generated levels and levels designed by humans to explore how perceptions of the creator relate to players'overall experience of gameplay. Players could not reliably identify the level's creator, yet their experiences were strongly linked to their beliefs about that creator rather than the actual truth. Levels believed to be human-made were rated as more fun and aesthetically pleasing. In contrast, those believed to be AI-generated were rated as more frustrating and challenging. This negative bias appeared spontaneously without knowing the levels'creator and often was based on unreliable cues of"human-likeness."Our results underscore the importance of understanding perception biases when integrating generative systems into games.

Problem

• generative AI
• player perception
• game experience
• human-AI collaboration
• procedural content generation

Innovation

• generative AI
• player perception
• procedural content generation
• human-AI collaboration
• user experience bias