Boosting Adverse Weather Crowd Counting via Multi-queue Contrastive Learning

2024-08-12

arXiv.org

AI Summary

To address degraded crowd counting performance under adverse weather conditions—caused by inter-domain distribution shifts across weather types and severe class imbalance in weather categories within training data—this paper proposes a two-stage multi-queue contrastive learning framework. In the first stage, a novel multi-queue Momentum Contrast (MoCo) mechanism is introduced to learn weather-aware representations within the backbone network, effectively mitigating class imbalance. In the second stage, cross-domain representation transfer is performed to adapt robust features learned under adverse weather to the normal-weather domain, enabling high-accuracy and weather-robust counting. The method requires no additional annotations and is compatible with mainstream CNN architectures. On benchmark datasets, it achieves a 22% reduction in Mean Absolute Error (MAE) under adverse weather conditions, with only a 13% increase in computational overhead, establishing new state-of-the-art performance.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Currently, most crowd counting methods have outstanding performance under normal weather conditions. However, our experimental validation reveals two key obstacles limiting the accuracy improvement of crowd counting models: 1) the domain gap between the adverse weather and the normal weather images; 2) the weather class imbalance in the training set. To address the problems, we propose a two-stage crowd counting method named Multi-queue Contrastive Learning (MQCL). Specifically, in the first stage, our target is to equip the backbone network with weather-awareness capabilities. In this process, a contrastive learning method named multi-queue MoCo designed by us is employed to enable representation learning under weather class imbalance. After the first stage is completed, the backbone model is"mature"enough to extract weather-related representations. On this basis, we proceed to the second stage, in which we propose to refine the representations under the guidance of contrastive learning, enabling the conversion of the weather-aware representations to the normal weather domain. Through such representation and conversion, the model achieves robust counting performance under both normal and adverse weather conditions. Extensive experimental results show that, compared to the baseline, MQCL reduces the counting error under adverse weather conditions by 22%, while introducing only about 13% increase in computational burden, which achieves state-of-the-art performance.

Problem

• Addressing domain gap between adverse and normal weather images
• Mitigating weather class imbalance in training datasets
• Enhancing crowd counting accuracy under adverse conditions

Innovation

• Multi-queue MoCo for weather class imbalance
• Two-stage weather-aware representation learning
• Contrastive learning for domain gap reduction