Beyond Conversations: Spatially-Anchored Previews for Intent Disambiguation in LLM-Assisted Geometry Editing in Virtual Reality

2026-07-25

AI Summary

This work addresses the challenge of ambiguous user intent in large language model (LLM)-assisted geometric editing within virtual reality (VR), where vague natural language instructions often lead to misinterpretation and inefficient clarification dialogues. To overcome the limitations of traditional conversational disambiguation—which suffers from poor efficiency and suboptimal user experience—the study introduces, for the first time, spatially anchored visual previews into the LLM-driven VR editing pipeline. It proposes a hybrid disambiguation paradigm that synergistically integrates real-time graphical previews with interactive dialogue. User studies demonstrate that this approach significantly reduces the number of interaction turns, enhances interaction stability, and improves user satisfaction, thereby offering empirical evidence and actionable design guidance for the effective integration of LLMs in immersive environments.

Abstract

User intent disambiguation remains a key challenge in intelligent interactive systems. While they have been widely studied in dialogue systems in 2D interfaces, research on how intent disambiguation could be incorporated within Large Language Model (LLM) assisted editing workflows in immersive environments remains limited. Recent advances in LLMs create opportunities to leverage the immersive nature of virtual and augmented reality (VR/AR) environments to provide better disambiguation support. In this paper, we evaluate how traditional dialogue-based disambiguation can be augmented with spatially-anchored graphical previews to resolve ambiguous user commands in LLM-assisted parameter-driven editing workflows. A within-subjects study in which 24 participants completed complex geometry editing tasks in VR simulate scenarios where VR scenes are controlled by numerical parameters. Compared with the condition where disambiguation is not available, quantitative metrics and qualitative feedback indicate that a hybrid approach which combines clarification questions and graphical previews can support better interaction stability with fewer conversation rounds while improving user experience. These findings provide empirical evidence on the effectiveness of disambiguation methods in LLM-assisted editing of parameter-driven immersive scenes and inform design guidelines for future integration of LLMs in advanced VR/AR systems.

Problem

• intent disambiguation
• LLM-assisted editing
• virtual reality
• geometry editing
• spatially-anchored previews

Innovation

• spatially-anchored previews
• intent disambiguation
• LLM-assisted editing
• virtual reality
• parameter-driven geometry editing