Does Visual Information Play a Decisive Role in Vision-Language-Action Model Driving Behavior?

2026-05-29

AI Summary

Current vision-language-action (VLA) driving models lack a systematic understanding of how visual inputs influence driving behavior, and conventional evaluations relying on aggregate performance metrics fail to uncover fine-grained relationships between visual perception and behavioral output. This work proposes the first structured, multi-level visual perturbation framework that applies controlled disturbances to input visual signals at the channel, information, and structural levels. By integrating open-loop trajectory prediction with closed-loop safety simulation, the framework enables a systematic analysis of model behavioral responses under varying visual conditions. Experiments reveal a pronounced imbalance in VLA models’ reliance on visual information across different abstraction levels, with this dependency pattern shifting notably depending on the evaluation paradigm. These findings provide critical empirical insights for developing safer and more robust VLA-based driving systems.

Abstract

Vision-Language-Action (VLA) models have demonstrated promising capability in autonomous driving, highlighting the potential of unified multimodal architectures for jointly modeling perception and planning. However, how current VLA-based driving behavior is grounded in visual information remains poorly understood. Existing evaluation protocols mainly focus on aggregate performance metrics, lacking structured and practical diagnostics to quantify visual-behavior dependency. In this work, we introduce a structured multi-level visual perturbation framework to analyze visual-behavior dependency in VLA-based driving models systematically. The framework organizes controlled visual perturbations along three complementary dimensions: channellevel degradation, information-level disruption, and structurelevel modification. We apply it to VLA-based driving systems and evaluate behavioral responses under both open-loop trajectory prediction and interactive closed-loop safety evaluation. Experimental results reveal evaluation-dependent dependency patterns and uneven visual grounding across abstraction levels. These findings call for more structured analyses and principled design of VLA driving models to better understand how visual information shapes behavior and develop safer, more robust systems.

Problem

• Vision-Language-Action
• visual grounding
• autonomous driving
• visual-behavior dependency
• multimodal perception

Innovation

• visual perturbation
• vision-language-action models
• visual-behavior dependency
• autonomous driving
• multimodal grounding