FlashDexRetarget: Accelerating Dexterous Manipulation Data Generation through Multi-Motion Retargeting

2026-10-01

AI Summary

This study addresses the bottlenecks of low success rates and poor training efficiency in cross-embodiment action retargeting for dexterous manipulation data generation. To this end, it proposes a reinforcement learning-based retargeting framework that introduces a novel multimodal observation mechanism fusing point cloud and trajectory encodings, alongside a complementary reward function. By incorporating independent left- and right-hand Actor-Critic architectures with the FlashSAC algorithm, the method achieves decoupled policy optimization while substantially reducing computational overhead. Evaluated on a benchmark of 50 actions, the proposed approach attains a 90% success rate, outperforming sampling-based baselines by 2.5× and reducing computational costs by two orders of magnitude compared to reinforcement learning baselines, while demonstrating superior scaling stability.

Abstract

Human hand-object demonstrations offer a reusable source of dexterous robot manipulation data, but transferring them across embodiments requires physically feasible retargeting. Existing physics-based approaches face limitations in retargeting success, motion-specific training efficiency, or both. To address these limitations, we introduce FlashDexRetarget, an RL-based framework for high-success, efficient dexterous motion retargeting. To make the demonstrated interaction easier to learn, we combine object point-cloud observations, hand-object distance features, and future trajectory encodings with complementary rewards that supervise object motion and reference hand-object relationships. To further accelerate learning, we employ separate left- and right-hand actor critic networks and adapt the off-policy algorithm, FlashSAC to dexterous motion tracking. On a benchmark of 50 motions spanning single-object and two-object interactions, FlashDexRetarget achieves a 90% success rate, approximately 2.5x that of the evaluated sampling-based baselines, while requiring up to 100x less training compute than the evaluated RL-based baselines. Evaluations on both XHand and Sharpa Wave Hand show consistent gains, and component-wise ablations examine the contributions of our design choices. Beyond the 50-motion benchmark, experiments with 200, 500, and 1,000 motions demonstrate that our method remains stable at larger scales and produces successful retargeted motions more efficiently as the training set grows. Qualitative replay results using real-world-captured demonstrations further illustrate the applicability of our framework to recorded human manipulation. Videos and code are available at https://davian-robotics.github.io/FlashDexRetarget/

Problem

• dexterous manipulation
• motion retargeting
• human demonstrations
• embodiment transfer
• reinforcement learning

Innovation

• Dexterous Manipulation
• Motion Retargeting
• Reinforcement Learning
• Off-policy Algorithm
• Scalability