Genie Sim PanoWorld: An Infinite Indoor 3D World Generation Pipeline via Panoramic Scene Modeling and Simulation

2026-07-29

AI Summary

This work proposes a two-stage feedforward method that generates high-fidelity, freely navigable indoor 3D scenes from a single 360° panoramic image without per-scene optimization or multi-view inputs. First, an SE(3) trajectory is planned via NavMesh, and geometric warping conditions guide a latent video diffusion model to produce controllable panoramic videos in just four denoising steps—without classifier-free guidance (CFG). Second, the generated video is reconstructed into a 3D Gaussian Splatting representation in a feedforward manner. By combining mixed short- and long-horizon trajectory training with self-consistency constraints, the approach balances global coherence over extended paths with fine local detail. The method outperforms geometry-conditioned baselines in both video generation and 3D reconstruction, and demonstrates zero-shot generalization to unseen scenes, directly yielding high-quality assets suitable for embodied AI simulation.

Abstract

We address the problem of reconstructing a high-fidelity, freely navigable 3D scene from a single $360^\circ$ panorama, without per-scene optimization or multi-view capture. Existing methods either lack metric trajectory control, which hinders reliable downstream 3D reconstruction, or struggle with large disocclusions under long-range camera motion while requiring high-end multi-GPU servers.We present Genie Sim PanoWorld, a two-stage feed-forward pipeline that bridges generation and reconstruction via an explicit, trajectory-controllable panoramic video. A NavMesh-planned $\mathrm{SE}(3)$ roaming trajectory is injected into a latent video diffusion model through dense geometry-warped conditioning; long--short trajectory mixed training and a self-consistency objective based on shortcut models together yield high-fidelity video in four CFG-free denoising steps. A feed-forward panoramic reconstructor then lifts the generated video into a high-fidelity 3D Gaussian scene that supports real-time, free-viewpoint roaming and can be directly used as a simulation-ready asset for embodied AI applications. Experiments show that Genie Sim PanoWorld outperforms geometry-conditioned baselines in both panoramic video generation and downstream 3D reconstruction, while generalizing zero-shot to unseen indoor scenes.

Problem

• 3D reconstruction
• 360-degree panorama
• free-viewpoint navigation
• indoor scene modeling
• single-image generation

Innovation

• panoramic scene modeling
• 3D Gaussian splatting
• trajectory-controlled generation
• feed-forward reconstruction
• embodied AI simulation