PlanCraft: Sketch, Refine, and Furnish for Architect-Inspired Progressive 3D Residential Scene Generation

2026-07-26

AI Summary

This work addresses a critical limitation in existing automated residential design methods, which neglect the progressive nature of architectural design and bypass the two-dimensional floor plan—a crucial spatial contract—leading to distorted layouts. To remedy this, the authors propose the first framework that emulates the architect’s iterative sketch-to-refinement workflow: SketchPlan first learns authentic drawing processes from real-world data; then, a coarse-to-fine diffusion model, PlanCraft-Diff, transforms low-fidelity sketches (as sparse as 25% completion) into precise vectorized floor plans; finally, a constraint-satisfying agent, PlanCraft-Agent, arranges furniture within well-defined room boundaries. By formally integrating progressive design thinking into 3D residential generation and establishing the 2D floor plan as an indispensable intermediate representation, the method achieves a 61.1% reduction in FID over the best prior 2D approach and scores 15 points higher in expert evaluations of spatial plausibility, substantially outperforming current systems.

Abstract

Two structural insights have been overlooked in automated residential floor plan generation. First, design is inherently progressive. Architects begin with rough strokes and refine them over time, whereas existing methods typically require their conditioning representation to be fully specified before generation, a fundamental mismatch with how design actually works. Second, the 2D floor plan is not an optional intermediate but an irreplaceable spatial contract. Once room boundaries, doors, and windows are fixed, furnishing reduces from open-ended spatial reasoning to bounded constraint satisfaction. Bypassing this contract, as existing 3D systems do by delegating layout to language models, yields overlapping rooms and implausible proportions; directly calling general-purpose language models likewise produces geometrically invalid layouts. Guided by these insights, we present PlanCraft. SketchPlan supplies the missing training signal by replaying the architect's drawing process on 80K real floor plans, producing partial sketches at every completeness level. PlanCraft-Diff progressively sharpens an incomplete sketch into a geometrically precise, vectorizable floor plan through a coarse-to-fine strategy. With the spatial contract established, PlanCraft-Agent then furnishes the scene within well-defined room boundaries. Experiments show that PlanCraft achieves a 61.1\% lower FID than the best existing 2D method and surpasses existing 3D systems by 15 points in expert-rated spatial rationality, with a sketch at only 25\% completion already outperforming all fully specified baselines.

Problem

• progressive design
• spatial contract
• 3D residential scene generation
• floor plan generation
• geometric validity

Innovation

• progressive generation
• spatial contract
• floor plan synthesis
• coarse-to-fine refinement
• 3D residential scene generation