VFAD: Variational Semantic Prompting Meets Frequency-Adaptive Representation Learning for Zero-Shot Anomaly Detection

2026-07-31

AI Summary

This work addresses the challenge of modeling diverse anomaly semantics and subtle local variations in zero-shot anomaly detection by proposing a unified framework, VFAD. Built upon CLIP, VFAD integrates a Variational Semantic Prompt Extractor (VSPE) and a Frequency-Adaptive Representation Aggregation (FARA) module. The VSPE employs a variational information bottleneck to regularize local semantic representations, while FARA enhances visual features through wavelet-based frequency decomposition and a mixture-of-experts mechanism. This design enables more precise cross-modal alignment and facilitates the learning of anomaly-sensitive features. Evaluated on 13 industrial and medical benchmarks, VFAD significantly outperforms existing methods, demonstrating superior generalization capability and localization accuracy.

Abstract

Zero-shot anomaly detection (ZSAD) aims to detect and localize anomalies in unseen categories without access to target-specific training data. Although recent CLIP-based methods have demonstrated promising generalization through vision-language alignment, they remain limited in capturing diverse anomaly semantics and subtle local variations. To address these limitations, we propose VFAD, a unified framework that combines variational semantic prompting with frequency-adaptive representation learning. Specifically, we introduce a Variational Semantic Prompt Extractor (VSPE), which adaptively aggregates anomaly-relevant local semantics from dense patch tokens and regularizes them through a variational information bottleneck, thereby incorporating fine-grained visual cues and enabling more precise cross-modal alignment. Furthermore, we develop a Frequency-Adaptive Representation Aggregation (FARA) module that leverages wavelet-based frequency decomposition and frequency-specific expert aggregation to enhance anomaly-discriminative visual representations. By jointly strengthening semantic guidance and visual representation learning, VFAD improves both anomaly discrimination and fine-grained localization. Extensive experiments on 13 industrial and medical benchmarks demonstrate that VFAD consistently outperforms existing state-of-the-art ZSAD methods across diverse anomaly scenarios. The code will be publicly available upon publication.

Problem

• Zero-Shot Anomaly Detection
• Anomaly Semantics
• Local Variations
• Vision-Language Alignment
• Unseen Categories

Innovation

• Variational Semantic Prompting
• Frequency-Adaptive Representation
• Zero-Shot Anomaly Detection
• Vision-Language Alignment
• Wavelet Decomposition