BuildingBlock: A Hybrid Approach for Structured Building Generation

2025-05-07

AI Summary

To address key challenges in 3D building generation—including limited diversity, structural incoherence, and hierarchical inconsistency—this paper proposes a two-stage hybrid generative framework. In the first stage, a diffusion-based Transformer models point-cloud layouts to ensure global geometric controllability. In the second stage, a large language model (LLM) interprets semantic constraints and infers rule-based hierarchical structures, which drive procedural content generation (PCG) for high-fidelity modeling. This work pioneers a synergistic paradigm integrating diffusion generation, LLM-driven reasoning, and PCG, enabling end-to-end architectural synthesis with semantic controllability, hierarchical traceability, and local editability. Evaluated on a custom-built architectural dataset and multiple benchmarks, our method achieves state-of-the-art performance, significantly improving diversity, structural plausibility, and hierarchical consistency. It further supports real-time interactive design and scalable deployment.

Technology Category

• Natural Language Processing: Generation
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Three-dimensional building generation is vital for applications in gaming, virtual reality, and digital twins, yet current methods face challenges in producing diverse, structured, and hierarchically coherent buildings. We propose BuildingBlock, a hybrid approach that integrates generative models, procedural content generation (PCG), and large language models (LLMs) to address these limitations. Specifically, our method introduces a two-phase pipeline: the Layout Generation Phase (LGP) and the Building Construction Phase (BCP). LGP reframes box-based layout generation as a point-cloud generation task, utilizing a newly constructed architectural dataset and a Transformer-based diffusion model to create globally consistent layouts. With LLMs, these layouts are extended into rule-based hierarchical designs, seamlessly incorporating component styles and spatial structures. The BCP leverages these layouts to guide PCG, enabling local-customizable, high-quality structured building generation. Experimental results demonstrate BuildingBlock's effectiveness in generating diverse and hierarchically structured buildings, achieving state-of-the-art results on multiple benchmarks, and paving the way for scalable and intuitive architectural workflows.

Problem

• Generating diverse structured 3D buildings efficiently
• Integrating generative models with procedural content generation
• Ensuring hierarchical coherence in architectural designs

Innovation

• Hybrid approach combining generative models, PCG, and LLMs
• Two-phase pipeline: Layout Generation and Building Construction
• Transformer-based diffusion model for globally consistent layouts