Scenix: Sparse-View 3D Scene Reconstruction via Executable Scene Programs

2026-08-07

AI Summary

This work addresses the challenge of reconstructing structured and editable 3D indoor scenes from a sparse set of uncalibrated RGB images, requiring joint inference of room layout, association of fragmented object observations, and recovery of globally consistent spatial structure. To this end, the authors propose Scenix, a novel framework that introduces executable scene programs as a structured representation, enabling end-to-end sparse-view reconstruction without manual annotations or precise 3D priors. The method integrates perception-driven asset instantiation, loop-closure-aware spatial optimization, and observation consistency supervision, effectively bridging multi-view geometry and semantic understanding. Experiments demonstrate that Scenix significantly outperforms existing approaches—particularly those relying on strong priors—on the XScene benchmark, real-world images, and out-of-distribution SpatialGen cases, excelling in scene structure prediction, object localization, and layout consistency.

Abstract

Synthesizing a structured and editable 3D indoor scene from a few uncalibrated RGB views requires more than generating high-quality individual assets: a system must infer the room structure, associate objects across incomplete observations, and recover a globally consistent spatial configuration. Previous methods mainly focus on 3D scene generation with text input or require continuous visual inputs with additional priors, \ e.g., human-annotated masks or accurate 3D layouts, which makes these methods labor demanding and hard to apply in general cases. We present \textsc{Scenix}, a sparse-view 3D scene reconstruction framework via executable scene programs, a structured representation that can be directly instantiated into editable 3D scenes. Given sparse views, \textsc{Scenix} predicts executable scene programs through perception-grounded asset instantiation and closed-loop spatial refinement. % We present \method, a framework that predicts an executable scene representation from sparse views and realizes it through perception-grounded asset instantiation and closed-loop spatial refinement. To support this task, we construct \dataset, a dataset of approximately 110,000 synthetic and real indoor scenes with multiview imagery, room structures, object-centric descriptions, and metric spatial annotations. We further introduce observation-consistent supervision that aligns each target scene with the visual evidence available in its input views. Experiments on held-out \textsc{XScene} scenes, real indoor images, and out-of-distribution SpatialGen cases evaluate structured scene prediction, object grounding, and spatial refinement.

Problem

• sparse-view reconstruction
• 3D scene reconstruction
• executable scene programs
• indoor scene understanding
• structure-from-motion

Innovation

• executable scene programs
• sparse-view reconstruction
• perception-grounded instantiation
• closed-loop spatial refinement
• observation-consistent supervision