AnchorDream: Repurposing Video Diffusion for Embodiment-Aware Robot Data Synthesis

2025-12-12

AI Summary

Real-world robotic demonstration data is scarce, while simulation data suffers from domain distortion and insufficient embodiment, hindering imitation learning. To address this, we propose an embodiment-aware video diffusion generation framework: leveraging robot kinematic rendering as a hard geometric anchor to guide a pre-trained video diffusion model in jointly synthesizing high-fidelity, diverse manipulation videos that strictly adhere to motion constraints. Our approach innovatively integrates spatiotemporal conditional control with cross-modal anchored distillation, eliminating action hallucination without requiring explicit environment modeling. Critically, it enables large-scale embodied video dataset construction from only a small number of teleoperated demonstrations. Experimental results demonstrate that downstream policies trained on the generated data achieve a 36.4% performance gain in simulation and nearly double the task success rate on real robots.

Technology Category

• Intelligent Robots: Embodied AI
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Diffusion Models for Vision

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

The collection of large-scale and diverse robot demonstrations remains a major bottleneck for imitation learning, as real-world data acquisition is costly and simulators offer limited diversity and fidelity with pronounced sim-to-real gaps. While generative models present an attractive solution, existing methods often alter only visual appearances without creating new behaviors, or suffer from embodiment inconsistencies that yield implausible motions. To address these limitations, we introduce AnchorDream, an embodiment-aware world model that repurposes pretrained video diffusion models for robot data synthesis. AnchorDream conditions the diffusion process on robot motion renderings, anchoring the embodiment to prevent hallucination while synthesizing objects and environments consistent with the robot's kinematics. Starting from only a handful of human teleoperation demonstrations, our method scales them into large, diverse, high-quality datasets without requiring explicit environment modeling. Experiments show that the generated data leads to consistent improvements in downstream policy learning, with relative gains of 36.4% in simulator benchmarks and nearly double performance in real-world studies. These results suggest that grounding generative world models in robot motion provides a practical path toward scaling imitation learning.

Problem

• Synthesizes diverse robot data from few demonstrations
• Addresses embodiment inconsistencies in generative models
• Improves imitation learning performance in real-world tasks

Innovation

• Repurposes video diffusion for robot data synthesis
• Anchors embodiment to prevent hallucination in motions
• Scales few demonstrations into large diverse datasets