MultiADS: Defect-aware Supervision for Multi-type Anomaly Detection and Segmentation in Zero-Shot Learning

2025-04-09

AI Summary

Industrial optical quality inspection urgently requires zero-shot methods capable of simultaneously recognizing and localizing multiple defect types (e.g., bending, scratches, cuts), whereas existing techniques are limited to binary anomaly detection and lack fine-grained type discrimination and pixel-level segmentation. This paper proposes the first zero-shot multi-defect-type segmentation framework: it constructs a text-guided joint vision–language feature space based on CLIP, incorporates a defect-aware supervision mechanism, and employs a lightweight linear projection layer to generate class-specific anomaly masks. The method enables parallel identification and pixel-accurate localization of multiple defects within a single image—without any target-category annotations. Evaluated on five benchmarks including MVTec-AD, it achieves significant improvements over state-of-the-art zero- and few-shot methods in both image-level detection and pixel-level segmentation performance.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multi-instance/Multi-view Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Precise optical inspection in industrial applications is crucial for minimizing scrap rates and reducing the associated costs. Besides merely detecting if a product is anomalous or not, it is crucial to know the distinct type of defect, such as a bent, cut, or scratch. The ability to recognize the"exact"defect type enables automated treatments of the anomalies in modern production lines. Current methods are limited to solely detecting whether a product is defective or not without providing any insights on the defect type, nevertheless detecting and identifying multiple defects. We propose MultiADS, a zero-shot learning approach, able to perform Multi-type Anomaly Detection and Segmentation. The architecture of MultiADS comprises CLIP and extra linear layers to align the visual- and textual representation in a joint feature space. To the best of our knowledge, our proposal, is the first approach to perform a multi-type anomaly segmentation task in zero-shot learning. Contrary to the other baselines, our approach i) generates specific anomaly masks for each distinct defect type, ii) learns to distinguish defect types, and iii) simultaneously identifies multiple defect types present in an anomalous product. Additionally, our approach outperforms zero/few-shot learning SoTA methods on image-level and pixel-level anomaly detection and segmentation tasks on five commonly used datasets: MVTec-AD, Visa, MPDD, MAD and Real-IAD.

Problem

• Detect and segment multiple defect types in products
• Identify distinct defect types like bent, cut, or scratch
• Perform zero-shot learning for anomaly detection and segmentation

Innovation

• Zero-shot learning for multi-type anomaly detection
• CLIP with linear layers for visual-text alignment
• Generates specific anomaly masks per defect type