Detecting Defects that Matter: An Application-Driven Benchmark for Anomaly Detection in Manufacturing and Retail Logistics (VAND 4.0 Challenge)
This study addresses the saturation and limited real-world applicability of existing anomaly detection benchmarks, which inadequately assess defect detection performance and computational efficiency in industrial and retail settings. To this end, we construct an application-driven benchmark featuring hidden test sets for these domains, propose a novel efficiency metric integrating performance with power consumption, and release Kaputt-Rare, a low-prevalence retail dataset. Comprehensive evaluations encompassing pixel-level segmentation and object detection are conducted using DINOv3, vision-language models (VLMs), and specialized supervised detectors. Results reveal that the best-performing method achieves only 57% SegF1 on industrial segmentation, while VLMs lag behind specialized models by approximately 28 AP in retail detection. These findings underscore the persistent challenges associated with efficiency optimization and rare defect detection in practical applications.