TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation

2026-06-15

AI Summary

Transferring human hand manipulation demonstrations to dexterous robotic hands often fails to preserve both hand poses and task-relevant hand-object contact structures, leading to degraded policy performance. This work proposes a general interaction-preserving retargeting framework that constructs a sparse interaction graph between hand and object keypoints and integrates distance-weighted Laplacian deformation optimization, orientation consistency constraints, kinematic limits, and penetration avoidance to achieve high-fidelity retargeting while maintaining task-critical contact topology. The method is the first to generalize across diverse retargeting scenarios using a single parameter set, achieving state-of-the-art contact accuracy and alignment on ContactPose, improving training success rate by 40.6 percentage points over baselines in the Pen-Spin task, and enabling zero-shot transfer to the Wuji Hand hardware for cube flipping and pen spinning.

Abstract

Human hand-object demonstrations provide dense reference motions for training dexterous manipulation reinforcement learning (RL) policies through reference tracking. However, to use such demonstrations for RL policy learning, retargeting must preserve hand pose and task-relevant hand-object contact structure. Otherwise, contact and feasibility artifacts can degrade downstream RL policy performance. We introduce TopoRetarget, an interaction-preserving retargeting framework that uses a single set of parameters across diverse retargeting conditions while maintaining task-relevant hand-object interaction and adapting human demonstrations to dexterous robot hands. The method constructs a sparse interaction graph over hand and object keypoints and optimizes distance-weighted Laplacian deformation with directional consistency, kinematic constraints, and penetration handling. Evaluations show that the generated references improve both interaction fidelity and policy learning: TopoRetarget achieves the best contact precision and alignment over all baselines on the ContactPose Dataset, improves Pen-Spin training success by 40.6 percentage points over the existing baseline methods, and enables zero-shot transfer to Wuji Hand hardware on cube reorientation and pen spinning.

Problem

• dexterous manipulation
• retargeting
• hand-object interaction
• reinforcement learning
• contact preservation

Innovation

• interaction-preserving retargeting
• dexterous manipulation
• sparse interaction graph
• Laplacian deformation
• zero-shot transfer