Text2Villa: Hierarchical Generation of 3D Indoor Environments with Physics-Aware Analysis-by-Synthesis

2026-07-19

AI Summary

Existing methods struggle to generate 3D indoor scenes from text that feature multiple rooms, vertical connectivity, and arbitrary polygonal boundaries, often lacking explicit modeling of physical constraints and consequently producing geometric interpenetration and floating artifacts. This work proposes a hierarchical generation framework: at the macro level, an autoregressive layout generator constructs multi-story building structures; at the micro level, a support-aware physical semantic scene graph (A-PSSG) is introduced, which—combined with an analysis-synthesis paradigm and a geometric engine—enables closed-loop optimization that explicitly encodes physical supportability and containment relations. Leveraging a multimodal large language model, the framework further performs physics-aware dynamic refinement. To our knowledge, this is the first method capable of generating high-fidelity, villa-scale, multi-level connected 3D indoor scenes with arbitrary boundaries, significantly outperforming existing approaches in both physical plausibility and geometric fidelity.

Abstract

Generating 3D indoor scenes from natural language holds tremendous potential, yet existing methods predominantly fail to generate multi-room structures with vertical connectivity and arbitrary polygonal boundaries. Furthermore, they lack a deep grounding in continuous 3D physical laws, leading to severe geometric penetrations and floating artifacts. In this work, we propose Text2Villa, a novel hierarchical generative framework. At the macro level, we construct a multi-story dataset to fine-tune an autoregressive layout generator, ensuring the direct parsing of text into 3D building foundations featuring polygonal boundaries and multi-story connectivity. To enforce physical laws during micro-level asset arrangement, we introduce the Affordance-driven Physical-Semantic Scene Graph (A-PSSG) to explicitly abstract physical affordances (such as support surfaces and containment cavities) into node attributes, establishing strict geometric and semantic edge constraints. Guided by the A-PSSG, we formulate scene instantiation as a constrained closed-loop optimization problem following the analysis-by-synthesis paradigm. By integrating an underlying geometric collision detection engine with the high-level semantic reasoning of multimodal large language models (MLLMs), our heuristic solver dynamically executes physics-aware actions under the observation-evaluation-modification mechanism to effectively resolve mesh collisions, floating artifacts, and fine-grained cavity containment failures. Extensive experiments demonstrate that Text2Villa outperforms previous methods across various metrics, robustly generating high-fidelity and physically plausible villa-level 3D environments from text, thereby providing a reliable and interactive 3D content foundation for downstream applications.

Problem

• 3D indoor scene generation
• multi-room layout
• physical plausibility
• geometric penetration
• vertical connectivity

Innovation

• hierarchical generation
• physics-aware synthesis
• affordance-driven scene graph
• analysis-by-synthesis
• multimodal large language models