Injecting Explainability and Lightweight Design into Weakly Supervised Video Anomaly Detection Systems

2024-12-28

AI Summary

To address the poor real-time performance, model bloat, and weak interpretability of weakly supervised video anomaly detection in smart-city edge surveillance, this paper proposes a two-stage cross-modal video anomaly detection system (TCVADS). In Stage I, knowledge distillation is employed to construct a lightweight binary classifier for rapid coarse screening. In Stage II, CLIP is integrated with customized textual triplet contrastive learning to enable fine-grained, semantically interpretable multi-class anomaly identification. Methodologically, TCVADS innovatively unifies cross-modal alignment and model lightweighting without requiring frame-level annotations. Evaluated on multiple benchmarks, TCVADS achieves a 3.2× speedup in inference latency, reduces model parameters by 68%, and—uniquely under weak supervision—provides semantic-level anomaly attribution. These advances collectively meet critical edge-deployment requirements: low latency, high trustworthiness, and human-understandable decision rationale.

Technology Category

Computer Vision: Multi-modal VisionMachine Learning: Learning on the Edge & Model CompressionConstraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSystems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applicationsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Weakly Supervised Monitoring Anomaly Detection (WSMAD) utilizes weak supervision learning to identify anomalies, a critical task for smart city monitoring. However, existing multimodal approaches often fail to meet the real-time and interpretability requirements of edge devices due to their complexity. This paper presents TCVADS (Two-stage Cross-modal Video Anomaly Detection System), which leverages knowledge distillation and cross-modal contrastive learning to enable efficient, accurate, and interpretable anomaly detection on edge devices.TCVADS operates in two stages: coarse-grained rapid classification and fine-grained detailed analysis. In the first stage, TCVADS extracts features from video frames and inputs them into a time series analysis module, which acts as the teacher model. Insights are then transferred via knowledge distillation to a simplified convolutional network (student model) for binary classification. Upon detecting an anomaly, the second stage is triggered, employing a fine-grained multi-class classification model. This stage uses CLIP for cross-modal contrastive learning with text and images, enhancing interpretability and achieving refined classification through specially designed triplet textual relationships. Experimental results demonstrate that TCVADS significantly outperforms existing methods in model performance, detection efficiency, and interpretability, offering valuable contributions to smart city monitoring applications.

Problem

• Weakly-Supervised Learning
• Video Anomaly Detection
• Smart City Surveillance

Innovation

• TCVADS
• Weakly-Supervised Learning
• Text-Image Fusion