Technology Category
Computer Vision: Diffusion Models for VisionNatural Language Processing: GenerationMachine Learning: Large Multimodal Models (LMMs)
STAY Diffusion: Styled Layout Diffusion Model for Diverse Layout-to-Image Generation
🤖 AI Summary
To address the challenges of synthesizing complex scenes and achieving fine-grained style control in layout-to-image (L2I) generation, this paper proposes a controllable diffusion-based generation framework. Our method introduces two key innovations: (1) Edge-Aware Normalization (EA-Norm), which incorporates edge-structure priors into the denoising process to enhance layout fidelity; and (2) Styled-Mask Attention, which explicitly models object-level semantic relationships while enabling independent, disentangled style control—ensuring both inter-object consistency and per-object stylistic customization. To our knowledge, this is the first diffusion-based L2I approach that jointly models global layout constraints and local semantic-style interactions. Extensive experiments on COCO-Stuff and Visual Genome benchmarks demonstrate significant improvements over state-of-the-art methods across layout fidelity, style controllability, and relational reasoning capability. Generated images exhibit higher visual fidelity and enhanced diversity.
Technology Category
Application Category
📝 Abstract
In layout-to-image (L2I) synthesis, controlled complex scenes are generated from coarse information like bounding boxes. Such a task is exciting to many downstream applications because the input layouts offer strong guidance to the generation process while remaining easily reconfigurable by humans. In this paper, we proposed STyled LAYout Diffusion (STAY Diffusion), a diffusion-based model that produces photo-realistic images and provides fine-grained control of stylized objects in scenes. Our approach learns a global condition for each layout, and a self-supervised semantic map for weight modulation using a novel Edge-Aware Normalization (EA Norm). A new Styled-Mask Attention (SM Attention) is also introduced to cross-condition the global condition and image feature for capturing the objects' relationships. These measures provide consistent guidance through the model, enabling more accurate and controllable image generation. Extensive benchmarking demonstrates that our STAY Diffusion presents high-quality images while surpassing previous state-of-the-art methods in generation diversity, accuracy, and controllability.
Problem
Research questions and friction points this paper is trying to address.
Generates photo-realistic images from coarse layout inputs.
Provides fine-grained control over stylized objects in scenes.
Enhances image generation diversity, accuracy, and controllability.
Innovation
Methods, ideas, or system contributions that make the work stand out.
STAY Diffusion model for layout-to-image synthesis
Edge-Aware Normalization for semantic map modulation
Styled-Mask Attention for cross-conditioning image features
🔎 Similar Papers
No similar papers found.
