LayoutCoT: Unleashing the Deep Reasoning Potential of Large Language Models for Layout Generation

2025-04-15

AI Summary

Existing conditional layout generation methods either rely heavily on large-scale annotated datasets and fine-tuning—suffering from poor generalizability—or leverage large language models (LLMs) via in-context learning, yet are constrained by limited spatial reasoning capabilities and simplistic ranking mechanisms, hindering sustained high-quality layout generation. This paper proposes a training-free zero-shot framework that synergistically integrates Layout-aware Retrieval-Augmented Generation (RAG) with iterative Chain-of-Thought (CoT) reasoning. Our approach employs structured prompting, layout serialization encoding, example retrieval, and multi-step refinement to elicit deep structural reasoning from general-purpose LLMs. To our knowledge, this is the first work demonstrating that general LLMs, under structured guidance, can surpass specialized reasoning models (e.g., DeepSeek-R1). Evaluated across five public benchmarks and three layout generation tasks, our method achieves state-of-the-art performance, with zero-shot layout quality significantly outperforming existing generative and training-free approaches.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Conditional layout generation aims to automatically generate visually appealing and semantically coherent layouts from user-defined constraints. While recent methods based on generative models have shown promising results, they typically require substantial amounts of training data or extensive fine-tuning, limiting their versatility and practical applicability. Alternatively, some training-free approaches leveraging in-context learning with Large Language Models (LLMs) have emerged, but they often suffer from limited reasoning capabilities and overly simplistic ranking mechanisms, which restrict their ability to generate consistently high-quality layouts. To this end, we propose LayoutCoT, a novel approach that leverages the reasoning capabilities of LLMs through a combination of Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) techniques. Specifically, LayoutCoT transforms layout representations into a standardized serialized format suitable for processing by LLMs. A Layout-aware RAG is used to facilitate effective retrieval and generate a coarse layout by LLMs. This preliminary layout, together with the selected exemplars, is then fed into a specially designed CoT reasoning module for iterative refinement, significantly enhancing both semantic coherence and visual quality. We conduct extensive experiments on five public datasets spanning three conditional layout generation tasks. Experimental results demonstrate that LayoutCoT achieves state-of-the-art performance without requiring training or fine-tuning. Notably, our CoT reasoning module enables standard LLMs, even those without explicit deep reasoning abilities, to outperform specialized deep-reasoning models such as deepseek-R1, highlighting the potential of our approach in unleashing the deep reasoning capabilities of LLMs for layout generation tasks.

Problem

• Enhancing layout generation with deep reasoning in LLMs
• Overcoming training data dependency in layout generation methods
• Improving semantic coherence and visual quality in layouts

Innovation

• Uses Retrieval-Augmented Generation for layout
• Employs Chain-of-Thought for iterative refinement
• Serializes layouts for LLM processing