DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation

2026-08-06

AI Summary

This work addresses the limited generalization of existing vision–language–action (VLA) models across heterogeneous robot embodiments, primarily hindered by insufficient exploitation of shared dynamical priors and reliance on manual action-space alignment. The authors propose DyPES-VLA, the first framework to jointly model cross-embodiment shared dynamics and embodiment-specific control policies. It leverages a vision–language model-driven future prediction objective to learn a shared dynamical representation from multi-source data and introduces embodiment-specific Mixture-of-Experts (MoE) action heads that directly output native control commands without manual alignment. Shared attention layers capture universal temporal structures, while dedicated experts adapt to diverse kinematic constraints. Evaluated on LIBERO, RoboCasa-GR1, and RoboTwin 2.0, DyPES-VLA achieves success rates of 98.0%, 59.25%, and 89.02%, respectively, substantially outperforming current state-of-the-art methods.

Abstract

Vision-Language-Action (VLA) models have become a powerful paradigm for robot manipulation, but training a single generalist policy for heterogeneous robot embodiments remains an open problem. Existing methods have two main limitations. First, they underuse dynamics priors shared across diverse visual and interaction data, limiting cross-embodiment transfer. Second, they require extensive manual preprocessing to convert embodiment-specific actions into a common format. To overcome these limitations, we propose DyPES-VLA, a cross-embodiment VLA that learns shared Dynamics Priors and Embodiment-Specific control. First, we learn shared dynamics priors by training the vision-language model (VLM) with a future-prediction objective on cross-embodiment data, driving the shared query representation to capture object motion, contact, and interaction-induced scene changes. Second, an embodiment-specific Mixture-of-Experts (MoE) action head translates these shared dynamics priors into executable controls directly in each embodiment's native action space, without manually pre-aligning heterogeneous actions into a common format. This head shares attention layers to capture common temporal action structures, while its embodiment-specific feed-forward experts resolve the unique kinematic constraints and control semantics of distinct embodiments. As a generalist policy, our \ourmethod achieves state-of-the-art performance across simulation and real-world evaluations, reaching 98.0% success on LIBERO, 59.25% on RoboCasa-GR1, and 89.02% on RoboTwin~2.0.

Problem

• cross-embodiment manipulation
• vision-language-action models
• shared dynamics priors
• embodiment-specific control
• generalist policy

Innovation

• Dynamics Priors
• Embodiment-Specific Control
• Mixture-of-Experts
• Cross-Embodiment Manipulation
• Vision-Language-Action