🤖 AI Summary
This study addresses the challenge of translating users’ intuitive spatial expressions into executable constraints for controllable, collaborative generative 3D design. To this end, we propose an XR system leveraging the Apple Vision Pro that integrates spatial sketches drawn with a Logitech Muse 3D pen and spoken language prompts, jointly encoding them for the first time as actionable constraints for an AI generative model (Meshy). The approach enables multiple users to synchronously collaborate and iteratively refine designs within a shared 3D space. Experimental results demonstrate the effectiveness of this workflow in enhancing design intuitiveness and fostering group consensus, while also highlighting the need for further improvements in generative efficiency and clarity of system feedback.
📝 Abstract
We present SpatialPrompt, an Extended Reality(XR) system that turns spatial sketches into executable constraints for controllable 3D generation. Users draw rough structures with a 3D pen and add voice prompts for semantic and stylistic intent. The system supports iterative refinement and synchronous co-creation in shared space with color-coded contributions. Implemented on Apple Vision Pro with Logitech Muse and Meshy, a heuristic evaluation suggests that the workflow is intuitive and supports shared understanding in collaborative creation, while revealing needs for faster generation and clearer feedback.