FedVAR: Prototype-Aligned Federated Framework for Video Anomaly Recognition

2026-08-07

AI Summary

This work addresses the semantic misalignment in federated video anomaly detection caused by data heterogeneity, particularly the inconsistent client-wise interpretations of “anomaly” under fine-grained categories. To resolve this, the authors propose a weakly supervised federated learning framework that introduces, for the first time, a prototype alignment mechanism into federated video anomaly detection. By leveraging vision-language models to construct shared semantic anchors, the framework harmonizes clients’ feature representations of “normal” and “anomalous” instances. Additionally, it integrates prompt learning to enable efficient recognition of novel anomaly classes. Extensive experiments demonstrate that the proposed method significantly outperforms existing federated approaches across diverse non-IID settings, unseen domains, and new anomaly categories, confirming its robustness and effectiveness in distributed cyber-physical systems.

Abstract

In the era of Industrial Internet of Things (IIoT) and Cyber-Physical Systems (CPS), Federated Learning (FL) offers a promising decentralized intelligence paradigm for Video Anomaly Recognition (VAR). This task is vital for maintaining high-fidelity Digital Twins and ensuring safety in mission-critical environments. However, the inherent data heterogeneity across distributed edge clients leads to a fundamental challenge known as semantic misalignment, where clients learn divergent feature representations of "normal" and "abnormal" events. The problem becomes particularly pronounced in VAR, where the presence of diverse and fine-grained anomaly categories leads each client to develop distinct semantic interpretations of abnormality. Existing federated methods primarily focus on binary anomaly detection and fail to address this misalignment, preventing effective fine-grained recognition. In this paper, we introduce FedVAR, a weakly-supervised FL framework explicitly designed for VAR. Leveraging the rich representations of Vision-Language Models (VLMs), FedVAR employs a prototype-based alignment mechanism that creates a shared semantic anchor for all clients to re-center and align their visual and textual feature spaces. This process enforces a consistent representation of "normality" across the decentralized network, directly mitigating semantic misalignment and enabling robust prompt-learning of anomaly direction vectors with minimal communication overhead. We conduct extensive experiments on challenging benchmarks under various non-IID data partitioning schemes, unseen domains, and novel anomaly classes. The results demonstrate that FedVAR consistently outperforms state-of-the-art federated baselines, establishing a robust framework for distributed intelligence in video-based CPS.

Problem

• Federated Learning
• Video Anomaly Recognition
• Semantic Misalignment
• Data Heterogeneity
• Fine-grained Recognition

Innovation

• Federated Learning
• Video Anomaly Recognition
• Prototype Alignment
• Vision-Language Models
• Semantic Misalignment