🤖 AI Summary
This study addresses the critical limitation of current video anomaly detection methods, which perform adequately within the same scene but suffer severe performance degradation when deployed across different scenes—a shortcoming obscured by the prevailing evaluation practice of reporting in-dataset AUC. The authors conduct the first systematic audit and quantification of this cross-domain performance collapse, proposing an unsupervised normality model built upon frozen off-the-shelf visual features (e.g., CLIP, DINOv2, ResNet-50, EfficientNet-B0) combined with nearest-neighbor search and PaDiM-style Mahalanobis distance for anomaly scoring. Experiments reveal that while intra-scene average AUC reaches 0.704, it plummets to 0.499—near random—in cross-scene settings, with DINOv2, the best-performing backbone, exhibiting the most pronounced drop and yielding a practical false alarm rate as high as 31,931 alerts per hour, thereby challenging the validity of existing evaluation paradigms.
📝 Abstract
Automated "suspicious behavior" flagging is a headline promise of AI surveillance, and the field reports high frame-level ROC-AUC on standard video anomaly detection benchmarks. Those numbers are measured by training and testing on the same camera and scene. We audit what happens when that assumption is dropped. We build an unsupervised normality model from the all-normal training frames of one dataset, using frozen off-the-shelf embeddings (CLIP, DINOv2, ResNet-50, EfficientNet-B0) and a nearest-neighbour distance, and score the test frames of the same and of other datasets. Across 4 real datasets (UCSD Ped1, UCSD Ped2, CUHK Avenue, ShanghaiTech) and 4 backbones, same-dataset AUC averages 0.704 but cross-dataset AUC averages 0.499, which is chance: a detector calibrated on one scene is no better than a coin flip on another, and in several pairs it is below chance. The strongest backbone makes this worse, not better: DINOv2 has the best same-dataset AUC (up to 0.901 on Ped2) and the largest cross-dataset drop. The collapse is not an artefact of the scoring rule: replacing the nearest-neighbour detector with a PaDiM-style Mahalanobis detector reproduces it almost exactly (cross-dataset gap 0.202 versus 0.208). Even at a favourable operating point the false-alarm rate is on the order of 31,931 per hour. We conclude that the benchmark numbers quoted for surveillance anomaly detection describe a calibrated laboratory setting and overstate deployable reliability by a wide margin, and we release the code that reproduces every number.