V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure

2026-07-23

AI Summary

This work addresses the critical issue of safety alignment failures in video large language models (VLLMs) when confronted with harmful videos paired with benign queries, which leads to anomalously high attack success rates. We propose V-DEAL, a three-tier diagnostic framework that, for the first time, attributes this vulnerability to insufficient coupling between visual understanding and refusal mechanisms. By quantifying refusal tendencies embedded in hidden states, we reveal that refusal signals triggered by visual comprehension are significantly weaker than those from textual understanding. Building on this insight, we design a prompt-injection intervention that requires no model fine-tuning. Evaluated across six state-of-the-art VLLMs, our method reduces the average attack success rate from 48.33% to 0.09%—a 48.24 percentage-point drop—matching the efficacy of fine-tuning-based approaches.

Abstract

As Video Large Language Models are increasingly deployed in real-world applications, ensuring their safety alignment has become critical. Counterintuitively, we find that harmful videos paired with benign queries achieve higher attack success rates than the same videos paired with explicitly harmful queries. To understand the underlying mechanism of this vulnerability, we present V-DEAL, a three-level diagnostic framework that jointly analyzes this failure across model behaviour, understanding, and internal representations. By progressively ruling out perception failure and quantifying the model's internal refusal tendency, V-DEAL provides a new diagnostic perspective for analyzing the underlying mechanism of the observed vulnerability. We tested six Video LLMs on three public benchmarks and observed that models correctly recognize harmful video content with over 81\% accuracy, yet the average attack success rate still reaches 48.33\% under the condition pairing harmful videos with benign queries. Hidden-state analysis further shows that visual understanding activates a weaker refusal tendency than textual understanding. Furthermore, we introduce a prompt injection intervention method that reduces attack success rates by an average of 48.24 percentage points and achieves performance comparable to prior fine-tuning-based methods, providing an effective and practical means to address such safety risks in Video LLMs.

Problem

• Video LLM safety
• alignment failure
• understanding-refusal coupling
• adversarial vulnerability
• safety de-calibration

Innovation

• Video LLM safety
• understanding-refusal coupling
• diagnostic framework
• prompt injection
• alignment failure