ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining

2025-11-07

AI Summary

To address the misalignment between ImageNet-pretrained features and industrial anomaly detection—stemming from objective mismatch and distribution shift—this paper proposes the first representation pretraining framework specifically designed for anomaly detection. Leveraging the large-scale industrial anomaly dataset RealIAD, we employ self-supervised contrastive learning with a novel dual-guided contrastive loss that explicitly maximizes both angular and norm-based separability between normal and anomalous samples in feature space. Additionally, residual feature modeling is introduced to enhance cross-domain generalization. Extensive experiments across five benchmark datasets and five backbone architectures demonstrate consistent and significant performance gains—achieved merely by replacing the embedding layer—validating the discriminability, universality, and practical utility of the proposed pretrained representations.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Representation Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

The current mainstream and state-of-the-art anomaly detection (AD) methods are substantially established on pretrained feature networks yielded by ImageNet pretraining. However, regardless of supervised or self-supervised pretraining, the pretraining process on ImageNet does not match the goal of anomaly detection (i.e., pretraining in natural images doesn't aim to distinguish between normal and abnormal). Moreover, natural images and industrial image data in AD scenarios typically have the distribution shift. The two issues can cause ImageNet-pretrained features to be suboptimal for AD tasks. To further promote the development of the AD field, pretrained representations specially for AD tasks are eager and very valuable. To this end, we propose a novel AD representation learning framework specially designed for learning robust and discriminative pretrained representations for industrial anomaly detection. Specifically, closely surrounding the goal of anomaly detection (i.e., focus on discrepancies between normals and anomalies), we propose angle- and norm-oriented contrastive losses to maximize the angle size and norm difference between normal and abnormal features simultaneously. To avoid the distribution shift from natural images to AD images, our pretraining is performed on a large-scale AD dataset, RealIAD. To further alleviate the potential shift between pretraining data and downstream AD datasets, we learn the pretrained AD representations based on the class-generalizable representation, residual features. For evaluation, based on five embedding-based AD methods, we simply replace their original features with our pretrained representations. Extensive experiments on five AD datasets and five backbones consistently show the superiority of our pretrained features. The code is available at https://github.com/xcyao00/ADPretrain.

Problem

• Addresses suboptimal ImageNet features for anomaly detection tasks
• Develops specialized pretraining for industrial anomaly detection scenarios
• Solves distribution shift between natural and industrial image data

Innovation

• Anomaly representation pretraining on industrial datasets
• Angle- and norm-oriented contrastive losses for feature separation
• Class-generalizable residual features to reduce distribution shift