TacOT: Learning Contact-Rich Dexterous Manipulation from Human Demonstrations via Tactile-Guided Optimal Transport

2026-10-03

AI Summary

This study addresses the unreliability of contact state correspondence during human-to-robot demonstration transfer, which arises from visual or kinematic similarities. To overcome this limitation, we propose TacOT, a framework that leverages tactile dynamics as semantic signals to disambiguate distinct contact states under visually similar motions. By integrating soft optimal transport with an action-tactile dynamic time warping alignment strategy within the representation space, TacOT enables efficient transfer of dexterous manipulation skills without requiring predefined frame-level correspondences. Evaluated across four real-world tasks, the proposed approach improves in-distribution success rates by 17% and out-of-distribution transfer performance by 20%. These results demonstrate that TacOT significantly enhances contact dynamic consistency in human-robot interaction, offering a robust solution for tactile-guided policy transfer in complex manipulation scenarios.

Abstract

Learning contact-rich dexterous manipulation from human demonstrations provides a scalable source of interaction data, yet transferring such skills to robots remains challenging due to unreliable human--robot correspondence. Existing human-to-robot transfer methods typically rely on visual appearance or motion similarity, which may associate similar motions with different contact states and force patterns. Tactile dynamics provide interaction-aware cues to distinguish manipulation processes with similar motions but different contact states. We introduce tactile-guided optimal transport (TacOT), a framework for human-to-robot contact-rich manipulation. TacOT leverages action--tactile dynamic time warping to identify human--robot demonstration correspondences with consistent interaction dynamics and uses these correspondences to guide soft optimal transport alignment in a shared policy representation space. This enables human demonstrations to provide contact-rich supervision for robot policy learning without requiring predefined frame-level human--robot pairing. Across four real-world dexterous manipulation tasks, TacOT improves closed-loop success rates over action-guided OT by up to 17 points on in-distribution tasks and 20 points under targeted human-to-robot out-of-distribution transfer. Further analyses show that tactile-guided correspondence selects demonstration pairs with more consistent contact dynamics and produces latent representations that better reflect interaction-state evolution. These results demonstrate that tactile dynamics provide an effective semantic signal for establishing reliable human-to-robot correspondence in contact-rich dexterous manipulation.

Problem

• dexterous manipulation
• human-to-robot transfer
• tactile dynamics
• contact-rich manipulation
• human-robot correspondence

Innovation

• Tactile-Guided Optimal Transport
• Dynamic Time Warping
• Dexterous Manipulation
• Human-to-Robot Transfer
• Contact Dynamics