SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RL

2026-10-01

AI Summary

This study addresses the limitation of existing LLM-based 3D indoor scene generation in retaining user preferences across sessions, which leads to repetitive and physically fatiguing VR creation experiences. To overcome this, we propose a novel framework that extracts persistent spatial preferences from multimodal natural interactions and constructs a hierarchical constraint model integrating local functionality with global topology. Furthermore, a human-in-the-loop reinforcement learning mechanism is incorporated to dynamically update retrieval strategies, enabling adaptive VR scene generation. This approach effectively mitigates shallow feature bias while significantly reducing the frequency of user corrections and physical exertion. Ultimately, it yields geometrically robust spatial layouts that align closely with personalized user profiles, advancing the state of the art in user-centric immersive environment synthesis.

Abstract

While Large Language Models (LLMs) advance 3D indoor scene synthesis, current pipelines fail to retain user-specific preferences across sessions, making immersive authoring a repetitive and physically fatiguing process. We present SPHERE, an adaptive VR generation framework that transforms isolated synthesis into continuous human-AI co-creation. SPHERE extracts persistent spatial preferences from natural multimodal interactions (speech and controller edits). To ensure geometric resilience against spatial distortions, it abstracts these raw edits into hierarchical constraints modeling both local functional and global topological contexts. Furthermore, a human-in-the-loop reinforcement learning mechanism dynamically updates retrieval policies based on the user's final edited scenes. A mixed-design user study ($N=42$) and an offline ablation demonstrate that SPHERE significantly reduces corrective edits and physical demand, preventing bias toward shallow object-level traits to yield geometrically resilient, profile-aligned layouts. Ultimately, SPHERE demonstrates how capturing demonstrated spatial logic enables controlled spatial adaptation, establishing a reliable, governed human-AI collaboration framework for immersive authoring. Project page and source code will be available at: https://github.com/hyeonmin11/SPHERE

Problem

• 3D indoor scene generation
• spatial preference learning
• VR authoring
• human-AI co-creation
• large language models

Innovation

• Spatial Preference Learning
• Human-in-the-Loop Reinforcement Learning
• VR Scene Generation
• Hierarchical Constraints
• Multimodal Interaction