LIBERO-Occ: Evaluating and Improving Vision-Language-Action Models under Scene-Induced Occlusion via Viewpoint Imagination

2026-06-09

AI Summary

Existing vision-language-action (VLA) models suffer significant performance degradation in real-world scenarios involving object occlusion, yet current evaluation benchmarks lack systematic consideration of occlusion. To address this gap, this work presents the first systematic investigation into scene-induced occlusion effects on VLA models, introducing LIBERO-Occ—a novel occlusion-oriented benchmark—and proposing the View Imagination Mechanism (VIM). VIM enhances robust action decision-making without requiring additional cameras by generating complementary viewpoints from the occluded primary view and fusing observed and imagined visual information. Extensive experiments demonstrate that VIM consistently improves model performance across diverse tasks, occlusion types, and severity levels, validating the efficacy of view imagination for perceptual completion in embodied AI.

Abstract

Vision-Language-Action (VLA) models achieve strong performance on standard manipulation benchmarks, but most evaluations assume that task-relevant objects are fully visible. This assumption often fails in realistic settings, where occlusion makes manipulation partially observable. In this paper, we study \textit{scene-induced occlusion} as a fundamental challenge for VLA models and introduce \textbf{LIBERO-Occ}, an occlusion-oriented extension of LIBERO. Experiments show that state-of-the-art VLAs suffer substantial performance degradation under occlusion. To address this issue, we propose \textbf{Viewpoint Imagination (VIM)}, which generates a complementary view from an occluded primary observation and conditions action prediction on both observed and imagined evidence. VIM improves robustness across task suites, occlusion types, and severity levels without requiring additional cameras at deployment time, suggesting that viewpoint imagination is an promising mechanism for perception completion in partially observable manipulation. Our benchmark and corresponding code are available at: \href{https://github.com/litsh/Libero-Occ}{https://github.com/litsh/Libero-Occ}.

Problem

• scene-induced occlusion
• Vision-Language-Action models
• partially observable manipulation
• occlusion robustness

Innovation

• Viewpoint Imagination
• Scene-induced Occlusion
• Vision-Language-Action Models
• Perception Completion
• Partially Observable Manipulation