Technology Category
Computer Vision: Diffusion Models for VisionMachine Learning: Large Multimodal Models (LMMs)Natural Language Processing: Generation
Ultrasound Image Generation using Latent Diffusion Models
🤖 AI Summary
This work addresses the critical scarcity of ultrasound images for rare diseases in medical imaging. We propose the first latent diffusion model (LDM) framework specifically designed for ultrasound generation. Methodologically, we pioneer the adaptation of Stable Diffusion to the ultrasound domain via multi-stage fine-tuning—including training on the BUSI dataset and clinical prompt engineering—augmented by ControlNet-guided synthesis using segmentation maps to ensure precise anatomical structure and pathological feature control. Innovatively, we introduce an anatomical consistency constraint to enhance clinical plausibility. Generated images were evaluated in a double-blind assessment by three ultrasound specialists and one radiologist, all of whom consistently rated them as realistic and clinically credible. The source code and pretrained models are publicly released to facilitate rapid, community-driven construction of high-quality breast ultrasound datasets.
Technology Category
Application Category
📝 Abstract
Diffusion models for image generation have been a subject of increasing interest due to their ability to generate diverse, high-quality images. Image generation has immense potential in medical imaging because open-source medical images are difficult to obtain compared to natural images, especially for rare conditions. The generated images can be used later to train classification and segmentation models. In this paper, we propose simulating realistic ultrasound (US) images by successive fine-tuning of large diffusion models on different publicly available databases. To do so, we fine-tuned Stable Diffusion, a state-of-the-art latent diffusion model, on BUSI (Breast US Images) an ultrasound breast image dataset. We successfully generated high-quality US images of the breast using simple prompts that specify the organ and pathology, which appeared realistic to three experienced US scientists and a US radiologist. Additionally, we provided user control by conditioning the model with segmentations through ControlNet. We will release the source code at http://code.sonography.ai/ to allow fast US image generation to the scientific community.
Problem
Research questions and friction points this paper is trying to address.
Generate realistic ultrasound images
Fine-tune diffusion models on medical datasets
Enhance training for classification and segmentation models
Innovation
Methods, ideas, or system contributions that make the work stand out.
Latent Diffusion Models
Fine-tuning on BUSI
ControlNet for segmentation
🔎 Similar Papers
