Shape Your Body: Value Gradients for Multi-Embodiment Robot Design

2026-05-30

AI Summary

Traditional robot design typically requires running reinforcement learning separately for each morphology, resulting in low efficiency and poor generalization. This work proposes a unified framework for training policy and value functions via multi-embodiment reinforcement learning. For the first time, a frozen multi-embodiment value function is leveraged as a differentiable surrogate to directly optimize new morphologies through value gradients, eliminating the need for retraining. The method efficiently optimizes robot designs within a vast continuous design space encompassing 50 distinct morphologies and over 1,100 continuous parameters. Moreover, it identifies key design and control factors that critically influence performance, substantially enhancing the automation and generalization capabilities of robot design.

Abstract

We propose to turn generalist multi-embodiment value functions into reusable models for robot design. Instead of running a new reinforcement learning co-design loop for each robot, we first train an embodiment-aware policy and value function across many robot designs. After training, the frozen value function is used as a differentiable surrogate to optimize candidate embodiments through value gradients. We evaluate our approach across different robot design settings, from perturbed single robots to held-out robots across morphology classes, with single models trained on up to 50 robots and design spaces of over 1100 continuous embodiment parameters. Beyond optimizing complete embodiments, we show that value gradients can identify performance-limiting design and control parameters, enabling both the optimization and the analysis of new robot designs.

Problem

• multi-embodiment
• robot design
• value gradients
• embodiment optimization
• morphology

Innovation

• value gradients
• multi-embodiment
• robot design
• differentiable surrogate
• embodiment-aware policy