ArXiv

DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation

Authors
Junfeng Li, Junjie He, Zhide Zhong...
Categories
cs.RO
arXiv
https://arxiv.org/abs/2608.06374v1
PDF
https://arxiv.org/pdf/2608.06374v1

Brief

DyPES-VLA presents a cross-embodiment vision-language-action model that trains a vision-language model with a future-prediction objective to learn shared dynamics priors from heterogeneous visual and interaction data; an embodiment-specific Mixture-of-Experts action head (shared attention layers plus per-embodiment feed-forward experts) maps priors to native action spaces without manual alignment. It reports 98.0% on LIBERO, 59.25% on RoboCasa-GR1, 89.02% on RoboTwin 2.0. Full text not available.

Why it matters

DyPES-VLA reports state-of-the-art cross-embodiment manipulation results: 98.0% success on LIBERO, 59.25% on RoboCasa-GR1, and 89.02% on RoboTwin 2.0.

Key details

  • The method learns shared dynamics priors by training a vision-language model with a future-prediction objective on cross-embodiment data, driving shared query representations to capture object motion, contact, and interaction-induced scene changes.
  • An embodiment-specific Mixture-of-Experts (MoE) action head translates the shared dynamics priors directly into each embodiment's native action space without manual action pre-alignment; it shares attention layers to model temporal action structure while per-embodiment feed-forward experts handle unique kinematics and control semantics.
Source evidence

Abstract

Vision-Language-Action (VLA) models have become a powerful paradigm for robot manipulation, but training a single generalist policy for heterogeneous robot embodiments remains an open problem. Existing methods have two main limitations. First, they underuse dynamics priors shared across diverse visual and interaction data, limiting cross-embodiment transfer. Second, they require extensive manual preprocessing to convert embodiment-specific actions into a common format. To overcome these limitations, we propose DyPES-VLA, a cross-embodiment VLA that learns shared Dynamics Priors and Embodiment-Specific control. First, we learn shared dynamics priors by training the vision-language model (VLM) with a future-prediction objective on cross-embodiment data, driving the shared query representation to capture object motion, contact, and interaction-induced scene changes. Second, an embodiment-specific Mixture-of-Experts (MoE) action head translates these shared dynamics priors into executable controls directly in each embodiment's native action space, without manually pre-aligning heterogeneous actions into a common format. This head shares attention layers to capture common temporal action structures, while its embodiment-specific feed-forward experts resolve the unique kinematic constraints and control semantics of distinct embodiments. As a generalist policy, our \ourmethod achieves state-of-the-art performance across simulation and real-world evaluations, reaching 98.0% success on LIBERO, 59.25% on RoboCasa-GR1, and 89.02% on RoboTwin~2.0.