A Taxonomy of Confabulations and the Perception-Reality Gap in LLM-Assisted Immersive Scene Editing

2026-07-25

AI Summary

This study addresses the propensity of large language models (LLMs) to generate hallucinations during immersive 3D scene editing, which can lead to user frustration, loss of trust, and safety risks. Through an exploratory study with 24 non-expert users, the work presents the first systematic taxonomy of LLM-induced hallucinations in this context and introduces the “perception–reality gap” theory to explain users’ cognitive biases in perceiving such errors. The research identifies a saturation effect in users’ awareness of hallucinations under high cognitive load. Combining user surveys, qualitative analysis, and human–computer interaction experiments within a custom-built, LLM-integrated immersive editing system, the study quantifies the frequency and disruptive impact of different hallucination types, offering targeted mitigation strategies and design guidelines for future systems.

Abstract

Large language models (LLMs) are being increasingly integrated into immersive environments and design workflows, providing application prospects in areas such as rapid scene prototyping for non-expert users and scene understanding capabilities for accessibility design. While many workflows that incorporate LLMs in immersive spaces are proposed, such systems can exhibit errors, potentially resulting in frustration, loss of user trust, and compromised user safety. This paper studies the underexplored area of LLM confabulations in immersive 3D scene editing contexts. Through an exploratory study with 24 non-expert users, we construct a taxonomy of the different types of confabulation observed in LLM-assisted immersive 3D scene editing. We report their prevalence and disruptiveness, and define the construct perception-reality gap to help understand the gap between the actual and perceived occurrence of confabulations. We highlight the observed saturation of confabulation awareness under load and conclude by discussing design implications for confabulation mitigation in future LLM-assisted systems in immersive 3D scenes.

Problem

• confabulation
• perception-reality gap
• LLM-assisted immersive editing
• 3D scene editing
• user trust

Innovation

• confabulation
• perception-reality gap
• LLM-assisted design
• immersive 3D editing
• taxonomy