DexDirect: Direct Kinesthetic Arm Guidance for Efficient Dexterous Demonstration Collection

2026-07-30

AI Summary

This work addresses the high cost, low efficiency, and substantial cognitive load associated with collecting dexterous manipulation demonstration data by proposing an efficient teaching method that integrates six-degree-of-freedom gravity-compensated robotic arm direct dragging with monocular vision-based hand retargeting. The approach enables real-time mapping of the operator’s non-dominant hand motions onto a 13-degree-of-freedom dexterous hand, significantly improving data collection efficiency while reducing mental workload under low hardware cost. Experimental results demonstrate that the proposed method yields 17.2× and 3.2× more successful demonstrations compared to pure vision- and pose-tracking baselines, respectively. Furthermore, a diffusion policy trained on the collected data achieves a 90% success rate on a cube pick-and-place task.

Abstract

Scalable collection of dexterous manipulation demonstrations remains a major bottleneck for robot learning. High-fidelity interfaces often require costly hardware and extensive setup, while low-setup, low cost alternatives tend to provide less precise control and impose greater cognitive workload on operators. We present DexDirect, a direct kinesthetic arm guidance for efficient dexterous demonstration collection. The operator drags a 6-DoF gravity-compensated robot arm directly by a handle, while a single webcam retargets operator's other hand onto a 16 joints 13-DoF dexterous robot hand. User studies suggest DexDirect collects 17.2x and 3.2x more successful demonstrations compared to purely vision (AnyTeleop) and pose-tracking (TeleDex) baselines. An adapted NASA-TLX shows DexDirect greatly reduces mental demand, effort, and frustration, despite raising physical demand. A diffusion policy trained on DexDirect demonstrations reaches a 90% success rate on a cube pick-and-place task. These results suggest that direct kinesthetic arm guidance combined with vision-based hand retargeting provides an efficient low-setup and scalable interface for collecting dexterous manipulation demonstrations

Problem

• dexterous manipulation
• demonstration collection
• robot learning
• teleoperation
• kinesthetic guidance

Innovation

• kinesthetic guidance
• vision-based retargeting
• dexterous manipulation
• teleoperation interface
• demonstration collection