MAGIC: Mask-Guided Diffusion Inpainting with Multi-Level Perturbations and Context-Aware Alignment for Few-Shot Anomaly Generation

2025-07-03

AI Summary

In industrial visual inspection, the scarcity of anomalous samples hinders effective anomaly detection; existing few-shot anomaly generation methods struggle to simultaneously achieve background fidelity, precise mask alignment, and semantically plausible anomaly localization. To address this, we propose a multi-level perturbation and context-aware alignment framework built upon Stable Diffusion. Our approach enhances semantic controllability via Gaussian-guided prompt perturbation, ensures spatially accurate region alignment through mask-conditioned noise injection, and achieves semantically grounded anomaly placement using context-aware mask relocalization. To the best of our knowledge, this is the first method to jointly optimize all three criteria within a unified generative pipeline. As a result, it significantly improves both the realism and diversity of synthesized anomalies. Under the standard MVTec-AD evaluation protocol, our method outperforms state-of-the-art approaches in downstream anomaly detection performance.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Generation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Few-shot anomaly generation is emerging as a practical solution for augmenting the scarce anomaly data in industrial quality control settings. An ideal generator would meet three demands at once, namely (i) keep the normal background intact, (ii) inpaint anomalous regions to tightly overlap with the corresponding anomaly masks, and (iii) generate anomalous regions in a semantically valid location, while still producing realistic, diverse appearances from only a handful of real examples. Existing diffusion-based methods usually satisfy at most two of these requirements: global anomaly generators corrupt the background, whereas mask-guided ones often falter when the mask is imprecise or misplaced. We propose MAGIC--Mask-guided inpainting with multi-level perturbations and Context-aware alignment--to resolve all three issues. At its core, MAGIC fine-tunes a Stable Diffusion inpainting backbone that preserves normal regions and ensures strict adherence of the synthesized anomaly to the supplied mask, directly addressing background corruption and misalignment. To offset the diversity loss that fine-tuning can cause, MAGIC adds two complementary perturbation strategies: (i) Gaussian prompt-level perturbation applied during fine-tuning and inference that broadens the global appearance of anomalies while avoiding low-fidelity textual appearances, and (ii) mask-guided spatial noise injection that enriches local texture variations. Additionally, the context-aware mask alignment module forms semantic correspondences and relocates masks so that every anomaly remains plausibly contained within the host object, eliminating out-of-boundary artifacts. Under a consistent identical evaluation protocol on the MVTec-AD dataset, MAGIC outperforms previous state-of-the-arts in downstream anomaly tasks.

Problem

• Generate realistic anomalies without corrupting normal background regions
• Ensure synthesized anomalies precisely align with given anomaly masks
• Produce diverse anomalies from few examples in valid locations

Innovation

• Fine-tunes Stable Diffusion for precise inpainting
• Uses Gaussian and spatial noise for diversity
• Aligns masks contextually to avoid artifacts