AC-VLA: Robust Out-of-Distribution Action Execution via Compositional Learning

2026-07-17

AI Summary

This work addresses the limited out-of-distribution (OOD) generalization of existing vision-language-action (VLA) models, particularly their difficulty in handling novel compositions of familiar subtasks. To mitigate trajectory overfitting and perceptual shortcuts without altering the backbone architecture, the authors propose the AC-VLA framework, which integrates a plug-and-play compositional learning module with a state-aware asymmetric masking mechanism. The approach leverages large language model–driven instruction decomposition, proprioception-aware trajectory alignment, a hybrid training strategy, and state-conditioned masking. Evaluated on the LIBERO and LIBERO-OOD benchmarks, AC-VLA achieves an absolute performance gain of approximately 28% on OOD tasks while maintaining near-perfect in-distribution accuracy.

Abstract

Vision-Language-Action (VLA) models excel at end-to-end robotic manipulation but struggle with out-of-distribution (OOD) generalization when familiar sub-tasks are recombined in unseen configurations. We identify two mutually reinforcing failure modes: \emph{trajectory overfitting}, where models overfit to holistic trajectory patterns rather than compositional sub-skill semantics; and \emph{perceptual shortcut}, where action tokens over-rely on wrist-view textures at the expense of global spatial grounding. To address both, we introduce \textbf{AC-VLA}, a plug-and-play Action Compositional learning framework comprising two architecture-agnostic components: \textbf{(i)} a compositional learning module that uses an LLM-driven instruction decomposer and a proprioceptive trajectory aligner to generate dense sub-task supervision, followed by mixed training on complete demonstrations and decomposed data to endow the model with compositional generalization; and \textbf{(ii)} a state-conditioned asymmetric masking strategy that suppresses wrist-view inputs during closed-gripper phases, enforcing global semantic grounding. All components are architectural modification-free and directly integrable into any VLA backbone. Instantiated on $π_{0.5}$ and evaluated on LIBERO and LIBERO-OOD benchmarks, AC-VLA achieves a ~28% absolute improvement on compositional OOD tasks while maintaining near-perfect in-distribution performance.

Problem

• out-of-distribution generalization
• Vision-Language-Action models
• compositional generalization
• trajectory overfitting
• perceptual shortcut

Innovation

• compositional learning
• out-of-distribution generalization
• vision-language-action models
• asymmetric masking
• trajectory decomposition