SceneLCM: End-to-End Layout-Guided Interactive Indoor Scene Generation with Latent Consistency Model

2025-06-08

AI Summary

Existing indoor scene generation methods suffer from rigid editing capabilities, physical incoherence, single-room constraints, and poor material quality. This paper proposes an end-to-end editable indoor scene generation framework that synergistically integrates large language models (LLMs) with latent consistency models (LCMs). We introduce Consistency Trajectory Sampling (CTS), a novel theoretical framework ensuring convergence during sampling. A normal-aware cross-attention decoder enables joint geometric-textural optimization, while a multi-resolution texture field coupled with physics simulation supports text-driven layout generation, furniture synthesis, and real-time physically consistent interaction. Our method achieves one-click generation of complex single- and multi-room scenes, significantly outperforming state-of-the-art approaches: layout plausibility improves by 23.6%, material PSNR increases by 8.4 dB, and both editing flexibility and physical realism reach new highs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation
• Computer Vision: Diffusion Models for Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Our project page: https://scutyklin.github.io/SceneLCM/. Automated generation of complex, interactive indoor scenes tailored to user prompt remains a formidable challenge. While existing methods achieve indoor scene synthesis, they struggle with rigid editing constraints, physical incoherence, excessive human effort, single-room limitations, and suboptimal material quality. To address these limitations, we propose SceneLCM, an end-to-end framework that synergizes Large Language Model (LLM) for layout design with Latent Consistency Model(LCM) for scene optimization. Our approach decomposes scene generation into four modular pipelines: (1) Layout Generation. We employ LLM-guided 3D spatial reasoning to convert textual descriptions into parametric blueprints(3D layout). And an iterative programmatic validation mechanism iteratively refines layout parameters through LLM-mediated dialogue loops; (2) Furniture Generation. SceneLCM employs Consistency Trajectory Sampling(CTS), a consistency distillation sampling loss guided by LCM, to form fast, semantically rich, and high-quality representations. We also offer two theoretical justification to demonstrate that our CTS loss is equivalent to consistency loss and its distillation error is bounded by the truncation error of the Euler solver; (3) Environment Optimization. We use a multiresolution texture field to encode the appearance of the scene, and optimize via CTS loss. To maintain cross-geometric texture coherence, we introduce a normal-aware cross-attention decoder to predict RGB by cross-attending to the anchors locations in geometrically heterogeneous instance. (4)Physically Editing. SceneLCM supports physically editing by integrating physical simulation, achieved persistent physical realism. Extensive experiments validate SceneLCM's superiority over state-of-the-art techniques, showing its wide-ranging potential for diverse applications.

Problem

• Automated generation of interactive indoor scenes from user prompts
• Overcoming rigid editing constraints and physical incoherence in scene synthesis
• Enhancing material quality and multi-room scene generation capabilities

Innovation

• LLM-guided 3D spatial reasoning for layouts
• Consistency Trajectory Sampling for furniture generation
• Normal-aware cross-attention decoder for textures