Improving Combined Detection and Classification of TEM Defects via Mask-Conditioned Latent Diffusion Augmentation

2026-06-01

AI Summary

This study addresses the challenge of detecting and classifying defects in irradiated metallic alloys from transmission electron microscopy (TEM) images, which is hindered by the scarcity of high-quality annotated data. To overcome this limitation, the authors propose a generative data augmentation approach that requires no manual labeling. Specifically, they introduce a mask-conditioned latent diffusion model (LDM) capable of controllably generating realistic TEM images along with corresponding multi-class defect masks. These synthetic data are then used to train a Mask R-CNN for joint defect detection and classification. Experimental results demonstrate that, under few-shot settings with only 10–100 real annotated images, the proposed method improves the harmonic mean of F1 scores by up to 0.02, confirming its effectiveness and practical utility.

Abstract

Analyzing microstructural defects in transmission electron microscopy (TEM) images, particularly in irradiated metal alloys, is often limited by the availability of high-quality, labeled data. To address this, we introduce a generative data augmentation approach using a mask-conditioned latent diffusion model (LDM) for synthesizing realistic TEM images with controllable, automatically labeled multi-class defect masks. Without requiring manual annotations for generation, our method enables the creation of synthetic image-mask pairs by sampling distributions learned from experimental masks. These generated data were used to augment small experimental datasets of varying sizes (10, 50, and 100 labeled experimental images) to train a Mask Regional Convolutional Neural Network (R-CNN) model for defect detection and classification. Our results show that generative augmentation yields small overall model performance improvements, with up to a 0.02 gain in the harmonic mean of detection and classification F1 scores. However, we also find that the relative contributions to detection and classification improvement depend on the specific train/test data split. These findings highlight the potential of targeted generative models to enhance deep learning performance in data-scarce microscopy-based image quantification tasks.

Problem

• TEM defects
• data scarcity
• defect detection
• defect classification
• microscopy image analysis

Innovation

• mask-conditioned latent diffusion
• generative data augmentation
• TEM defect detection
• automatic mask labeling
• Mask R-CNN