Novel Anomaly Detection Scenarios and Evaluation Metrics to Address the Ambiguity in the Definition of Normal Samples

2026-04-08

AI Summary

This work addresses the practical challenge in industrial anomaly detection where “normal” samples often exhibit ambiguous definitions—such as tolerating minor defects or evolving quality standards—contrary to the common assumption that training data are perfectly normal. To bridge this gap, the study presents the first systematic formulation of this realistic setting, introduces tailored evaluation metrics, and proposes RePaste, a novel method that iteratively re-pastes image regions with high anomaly scores back into the input to adaptively enhance the model’s discrimination capability under ambiguous normality. Evaluated on a new benchmark derived from MVTec AD, RePaste achieves state-of-the-art performance under the proposed metrics while maintaining leading results in conventional AUROC and PRO scores, demonstrating its effectiveness and robustness in scenarios with ill-defined normal samples.

Technology Category

Computer Vision: Adversarial Attacks & RobustnessMachine Learning: Adversarial Learning & RobustnessData Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

Search and Retrieval-Augmented AI: Web evaluation methodologies and metricsWeb Mining and Content Analysis: Robustness and generalizability of Web mining methodsEconomics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

In conventional anomaly detection, training data consist of only normal samples. However, in real-world scenarios, the definition of a normal sample is often ambiguous. For example, there are cases where a sample has small scratches or stains but is still acceptable for practical usage. On the other hand, higher precision is required when manufacturing equipment is upgraded. In such cases, normal samples may include small scratches, tiny dust particles, or a foreign object that we would prefer to classify as an anomaly. Such cases frequently occur in industrial settings, yet they have not been discussed until now. Thus, we propose novel scenarios and an evaluation metric to accommodate specification changes in real-world applications. Furthermore, to address the ambiguity of normal samples, we propose the RePaste, which enhances learning by re-pasting regions with high anomaly scores from the previous step into the input for the next step. On our scenarios using the MVTec AD benchmark, RePaste achieved the state-of-the-art performance with respect to the proposed evaluation metric, while maintaining high AUROC and PRO scores. Code: https://github.com/ReijiSoftmaxSaito/Scenario

Problem

• anomaly detection
• normal sample ambiguity
• industrial inspection
• specification change
• evaluation metrics

Innovation

• anomaly detection
• ambiguous normality
• RePaste
• evaluation metrics
• industrial inspection