Problem
Research questions and friction points this paper is trying to address.
Vision-Language-Action policies
Latent world modeling
Long-horizon control
Future state prediction
PLaW-VLA: Predictive Latent World Modeling for Vision-Language-Action Policies
🤖 AI Summary
This study addresses the heavy computational burden of future state prediction and limited generalization in Vision-Language-Action (VLA) policies for long-horizon control by proposing a lightweight latent world model. The method models task-relevant latent states to circumvent low-level visual reconstruction, while integrating a hybrid Transformer architecture with structured causal attention to optimize action generation and enable efficient parallel prediction. Experimental results demonstrate that the proposed model achieves an inference latency merely 1/19th that of generative approaches. Furthermore, it outperforms reactive policies by 11.8% on the RoboTwin benchmark and yields a 1.77% improvement in zero-shot evaluations on LIBERO-Plus, significantly enhancing both control efficiency and robustness.
📝 Abstract
Learning to predict how the world evolves can provide vision-language-action (VLA) policies with predictive context for long-horizon control, but its effectiveness depends on what future representation is modeled and how it conditions action generation. We introduce PLaW-VLA, which models task-relevant future states in a pretrained prediction-oriented representation space, reducing the need to predict control-irrelevant visual details. Built on a Mixture-of-Transformers architecture, PLaW-VLA conditions action generation on observation history, current task semantics, and predicted future states through structured causal attention. Experiments show a +11.8 percentage-point (pp) gain over reactive policies on RoboTwin Hard Horizon III and a +1.77 pp gain over reconstruction-oriented latent prediction on zero-shot LIBERO-Plus, supporting improved long-horizon control and generalization under distribution shift, respectively. By avoiding low-level visual reconstruction, PLaW-VLA lowers the burden of future prediction, enabling a lightweight latent world model with parallel future prediction and about 1/19 the inference latency of generative world-action modeling at comparable policy performance.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Vision-Language-Action Policies
Latent World Model
Mixture-of-Transformers
Predictive Representation
Long-horizon Control
🔎 Similar Papers
No similar papers found.
