DensFiLM: Density-Conditioned Video Saliency for Crowd Scenes

2026-07-28

AI Summary

This work addresses the limitation of existing video saliency models, which employ a uniform gaze strategy and struggle to adapt to attentional variations across different crowd densities. To overcome this, the authors propose the first density-conditioned video saliency model by integrating a lightweight FiLM module into the bottleneck layer of a Video Swin Transformer. This module dynamically modulates feature channels through scaling and shifting based on crowd density embeddings, enabling adaptive modeling for both sparse and dense crowd scenes. The approach introduces only approximately 100K additional parameters and supports either ground-truth or predicted density labels. On the CrowdFix benchmark, it achieves an NSS of 1.434 and a CC of 0.517, surpassing ACLNet by over 14%. Ablation studies confirm that density conditioning yields substantial performance gains, with predicted density labels performing comparably to ground-truth ones.

Abstract

Video saliency models typically apply a single fixation strategy across crowd scenes, despite systematic changes in attention with crowd density. Sparse scenes encourage tracking individuals, whereas dense scenes shift attention toward collective motion and scene-level landmarks. We introduce DensFiLM, a density-conditioned video saliency model that inserts a lightweight Feature-wise Linear Modulation layer at the bottleneck of a Video Swin Transformer. A learned density embedding produces channel-wise scale and shift parameters, allowing the decoder to reconstruct saliency from features selected for each density regime. The module adds only ~100K parameters and can use either CrowdFix density labels or the model's own density prediction. On CrowdFix, DensFiLM achieves mean NSS 1.434 and CC 0.517 over four seeds, improving over ACLNet by 14.7% and 14.9%, respectively, while predicted-density conditioning matches oracle-label performance. Ablations show that explicit RAFT optical flow and larger temporal and social-force extensions provide no further improvement in this setting. In a centre-prior-subtraction diagnostic, density conditioning yields an NSS gain of 0.462 over the unconditioned backbone, compared with 0.124 under standard evaluation. These results show that lightweight bottleneck conditioning provides a more effective inductive bias than increasing model capacity for crowd-video saliency. Our code is available at https://github.com/aniskhan25/crowdfix-saliency.

Problem

• video saliency
• crowd scenes
• crowd density
• attention mechanism
• density conditioning

Innovation

• density-conditioned
• video saliency
• FiLM
• crowd scenes
• lightweight modulation