OTRetarget: Joint Robot and Object Motion Retargeting via Optimal Transport

2026-09-28

AI Summary

This study addresses the challenge of maintaining ground and object contacts during humanoid motion retargeting, where morphological discrepancies between source and target bodies often compromise interaction fidelity. To this end, it proposes a unified framework for jointly retargeting robot and multi-object motions by introducing a surface interaction representation and constrained inverse kinematics. By integrating entropy-regularized optimal transport, signed distances, and reinforcement learning-based whole-body policies, the approach enables simultaneous optimization of robot and object poses without requiring scene or demonstration scaling, effectively balancing contact preservation with motion style. Evaluated on the OMOMO dataset, the method achieves an interaction Jaccard score of 87% with a depth error of merely 8.7 mm. Furthermore, successful deployment on the physical G1 humanoid robot validates its effectiveness and generalization capability in complex interactive scenarios.

Abstract

Transferring human motion to humanoid robots requires adapting the demonstrated motion to the robot morphology while preserving interactions with the environment. This is particularly challenging for loco-manipulation tasks, where contacts with the ground and manipulated objects must remain consistent despite differences in body proportions. Yet, skeletal motion alone does not fully describe these interactions, and fixing object trajectories limits the adaptation to a new embodiment. In this paper, we introduce OTR ETARGET, a unified approach to jointly retarget robot and multi-object motion from human demonstrations. Our approach represents surface interactions through signed distances, closest surface points, and relative directions, and uses entropic optimal transport to transfer these quantities across human, robot, and object geometries. We incorporate the resulting interaction targets into a constrained inverse kinematics formulation that balances contact preservation with motion style and jointly optimizes robot and object poses at each frame. This formulation accommodates robot-object and object-object interactions without rescaling the scene or the demonstration. We validate the proposed approach on OMOMO, where it achieves a robot- object interaction Jaccard score of 87% and a depth error of 8.7 mm, compared with 28% and 29.3 mm for OmniRetarget. Finally, we demonstrate transfer to a physical G1 humanoid using whole-body policies trained with reinforcement learning on the retargeted references, across motions including two-handed box pick-and-place onto a table.

Problem

• motion retargeting
• humanoid robots
• loco-manipulation
• robot-object interaction
• optimal transport

Innovation

• Optimal Transport
• Motion Retargeting
• Inverse Kinematics
• Loco-manipulation
• Humanoid Robot