ContactFlow: A video action conditioning that transfers across embodiments

2026-07-29

AI Summary

Existing video world models struggle to capture physical contact constraints and rely on robot-specific action conditioning, hindering cross-embodiment transfer. To address this, this work proposes Contact Flow—a body-agnostic, universal action representation that encodes 3D contact point trajectories between the agent and objects, thereby unifying action conditions from human demonstrations and robot executions. Building upon this representation, we introduce a closed-loop planning framework—“propose–imagine–verify–execute”—that integrates a video-generative world model with visual-language model–based evaluation. Experiments demonstrate that our approach enables effective cross-embodiment action transfer on the DROID dataset and real-world tabletop manipulation tasks, generating physically plausible action predictions.

Abstract

World models offer a promising route toward robot planning by enabling agents to imagine and verify the consequences of actions before execution. However, current video-based world models often struggle to capture the physical constraints that govern manipulation, particularly contact. Further, their action conditioning is often constrained to specific embodiments such as parallel grippers. We propose \emph{Contact Flow}, an embodiment-agnostic action representation that encodes manipulation through the trajectory of 3D contact points between an actor and a target object. By discarding actor-specific appearance and kinematics, Contact Flow provides a shared conditioning signal for both human demonstrations and robotic execution. Therefore, we can train a large-scale video generative model on both human and robotic object interaction videos conditioned on Contact Flow, yielding a world model that predicts physically plausible manipulation outcomes. We integrate this model into a propose-imagine-verify-act pipeline, where generated rollouts are assessed by a vision-language model before execution. Experiments on the DROID dataset and real-world tabletop manipulation tasks demonstrate that Contact Flow enables transfer between human demonstrations and different robotic embodiments.

Problem

• world models
• action conditioning
• embodiment transfer
• contact constraints
• manipulation

Innovation

• Contact Flow
• embodiment-agnostic
• video world model
• 3D contact points
• cross-embodiment transfer