Does Adversarial Training Improve Generalization in Multi-View VLAs? Revealing and Mitigating View Collapse

2026-09-27

AI Summary

This study addresses the phenomenon of "view collapse" in multi-view Vision-Language-Action (VLA) models during adversarial training, which leads to spurious robustness improvements. We elucidate this failure mechanism and propose an analytical framework that decouples robust perception from robust fusion. Furthermore, we introduce a view-swapping intervention strategy to mitigate view-dependent bias. Extensive experiments encompassing pre-trained VLM adaptation, adversarial training, and evaluation on the LIBERO-Plus benchmark demonstrate that integrating view swapping significantly enhances model generalization under multi-camera viewpoint shifts, sensor noise perturbations, and initial state variations.

Abstract

Vision-language-action (VLA) models adapt pretrained vision-language models (VLMs) for closed-loop robot control, transferring their perceptual and semantic capabilities to action prediction. Despite strong in-distribution performance, however, VLAs often degrade under deployment shifts. Adversarial training (AT) offers a model-adaptive approach to robustness without explicitly anticipating individual shifts, but its effect on natural distribution-shift generalization in multi-view VLAs remains unclear. We study this question using a multi-view VLA directly adapted from a pretrained VLM and evaluate generalization across seven LIBERO-Plus shift axes. Direct AT substantially improves Camera Viewpoint and Sensor Noise, the two shifts affecting only the third-person view, yet produces mixed or negative effects on other shifts. Controlled view interventions reveal a surprising failure mode that we term view collapse: Direct AT can shift cross-view reliance so strongly that the policy becomes dominated by the wrist view. This exposes a \textit{robustness shortcut}: apparent robustness to a shifted view can arise from reduced use of that view rather than more robust perception of it. This motivates a distinction between robust perception, extracting reliable information under within-view shifts, and robust fusion, adapting reliance across views according to their reliability. To reduce fixed view reliance, we use a simple View Swap intervention and then re-evaluate AT. With View Swap, AT further improves Camera Viewpoint, Sensor Noise, and Robot Initial State, while its effects remain mixed on other shifts. Our results show that multi-view robustness requires separating improved perception from changes in cross-view reliance, and that AT provides selective rather than generic distribution-shift benefits.

Problem

• Vision-Language-Action Models
• Adversarial Training
• Multi-View Robustness
• View Collapse
• Distribution Shift Generalization

Innovation

• Adversarial Training
• Multi-View VLA
• View Collapse
• Robustness Shortcut
• View Swap