CSGen: A Multi-Domain Curvilinear Structure Generation Model via Hierarchical Multimodal Diffusion

2026-08-05

AI Summary

Controllable generation of images with precise curvilinear structures remains an open challenge in multimedia research. To address this, this work proposes CSGen—a hierarchical multimodal diffusion model that enables high-fidelity image synthesis through multi-stage conditional control, achieving accurate alignment with diverse control signals. The key contributions include the construction of a multidomain, multimodal dataset spanning five domains and seven annotation types; the design of a hierarchical progressive control strategy that decouples topological structure from visual appearance; and the introduction of a sparsity-aware loss reweighting mechanism to enhance learning of fine and weak structures. Experiments demonstrate that CSGen significantly outperforms existing methods in both structural accuracy and visual realism, effectively improves downstream segmentation performance, and maintains robustness across diverse prompting conditions.

Abstract

Curvilinear structure analysis is an important and fundamental task in multimedia. However, the controllable generation of images with precise curvilinear structure objects remains an open challenge. To address this, we propose CSGen, a hierarchical multimodal diffusion model that synthesizes high-fidelity images precisely aligned with multiple control conditions. The CSGen is built upon three key innovations: 1) We construct a multi-domain and multimodal dataset, including over 24K samples from 5 domains and 7 different types of annotations, to train the unified generation model. 2) We propose a novel hierarchical progressive control strategy that decouples topology clues from visual context by a phased signal injection, mitigating semantic drift while ensuring the topological integrity of sparse structures. 3) We design a sparsity-aware loss re-weighting mechanism to address the extreme sparsity of curvilinear structures, significantly enhancing the attention on thin and fragile structures during optimization. Extensive experiments demonstrate that CSGen generates images with superior structure accuracy and visual realism, significantly improving downstream segmentation performance while maintaining robustness across diverse prompts. Our results confirm CSGen as a scalable, data-centric paradigm for the analysis of complex curvilinear structures in diverse multimedia applications. Code and dataset are available at https://github.com/ShanZard/CSGen.

Problem

• curvilinear structure
• controllable generation
• image synthesis
• multimodal diffusion
• structure accuracy

Innovation

• hierarchical multimodal diffusion
• curvilinear structure generation
• progressive control strategy
• sparsity-aware loss
• multi-domain dataset