DualDiT: A Conditional Dual-Output Diffusion Transformer for Joint OCT Image and Segmentation Mask Generation

2026-07-31

AI Summary

This study addresses the scarcity of annotated data in joint generation of mouse ocular OCT images and their corresponding retinal layer segmentation masks by proposing a Conditional Dual-Output Diffusion Transformer (DualDiT). This work introduces diffusion Transformers to the joint synthesis of medical images and masks for the first time, overcoming limitations of conventional U-Net architectures. The method employs a pretrained VAE to encode both modalities into a shared latent space, where their representations are concatenated and jointly generated through a conditional diffusion mechanism. Experimental results demonstrate that DualDiT outperforms baseline models in terms of FID (56.14) and sFID (114.35), with 46% of synthesized samples misclassified as real by expert reviewers. Furthermore, the generated data significantly improves Dice and IoU scores in downstream segmentation tasks.

Abstract

Background and Objective: Generating realistic medical images with anatomically accurate segmentation masks helps address the shortage of annotated data in medical imaging, particularly in optical coherence tomography (OCT) of mouse eyes, where manual retinal layer delineation is labour-intensive due to tiny structures and required expertise, resulting in scarce datasets. While diffusion models perform well in medical image synthesis, joint image-mask generation has relied mainly on U-Net-based denoisers, leaving diffusion transformers largely unexplored. Methods: We propose a conditional dual-output Diffusion Transformer (DualDiT) for joint synthesis of OCT B-scans and segmentation masks of the upper retinal cell layers in ex vivo mouse retina. DualDiT encodes both modalities into a shared latent space via a pretrained VAE, concatenates their latent representations, and performs conditional diffusion over the joint tensor. We compared DualDiT against two adapted diffusion baselines: DDPM and LDM. Generative quality was assessed via Fréchet Inception Distance (FID) and spatial FID (sFID); practical utility via synthetic data augmentation for downstream U-Net segmentation; and perceptual realism via evaluation by three domain experts. Results: DualDiT achieved the best generative quality (FID 56.14, sFID 114.35), outperforming DDPM and LDM. Expert panels misclassified 46% of synthetic samples as real and 42% of real samples as synthetic. Adding DualDiT-generated images and masks improved Dice and IoU scores on a held-out segmentation test set. Conclusions: DualDiT shows that transformer-based diffusion models can effectively learn the joint distribution of OCT images and segmentation masks, surpassing DDPM- and LDM-based baselines in generative fidelity, downstream utility, and perceptual realism, highlighting its potential for data augmentation in annotation-scarce medical imaging.

Problem

• OCT image synthesis
• segmentation mask generation
• data scarcity
• medical image annotation
• joint image-mask generation

Innovation

• Diffusion Transformer
• joint image-mask generation
• medical image synthesis
• OCT
• data augmentation