ExoBridge: Learning a Bare Hand to Hand-Worn Exoskeleton Mapping through Human Limb Coupling

2026-10-05

AI Summary

This study addresses the challenge of acquiring kinematic and tactile supervision signals from bare-hand videos for dexterous robot learning. To this end, it proposes a bimanual synergy-based self-supervised bridging framework that leverages human limb coupling mechanisms. Specifically, a temporal vision model maps uninstrumented bare-hand videos to the kinematic and tactile states of a sensorized exoskeleton, enabling cross-modal contact detection and tactile intensity regression. Evaluated across four manipulation tasks, the proposed method achieves a contact detection AUROC of 0.916 and a Pearson correlation coefficient of 0.790 for tactile intensity prediction. These results effectively validate both the feasibility and precision of the proposed visual-to-physical state bridging approach for advancing dexterous robotic manipulation.

Abstract

Human video offers a scalable source of experience for dexterous robot learning, but obtaining motion and tactile supervision while preserving bare hand interaction remains challenging. We present ExoBridge, a framework that leverages human limb coupling to learn a bridging function from bare hand video to the motion and tactile state of a sensorized exoskeleton. Our central idea is to use coordinated bimanual behavior to connect an uninstrumented visual source with a measured manipulation interface. During collection, one hand remains bare and provides visual observations, while the opposite hand wears the exoskeleton and supplies synchronized motion and tactile measurements. These paired demonstrations train a temporal visual model to predict fingertip contact, continuous tactile intensity, and relative encoder motion from bare hand video alone. The exoskeleton defines an intermediate state space whose motion coordinates are linked to a dexterous robot hand through existing calibration. Evaluation on 1,215 demonstrations across four manipulation tasks uses held out collection sessions and yields a pooled any contact AUROC of 0.916 and a Pearson correlation of 0.790 for tactile intensity. The learned bridge also predicts relative changes in exoskeleton configuration from bare hand video. These results demonstrate that human limb coupling can turn exoskeleton measurements into supervision for bare hand video, establishing a learned bridge between human visual demonstrations and a robot oriented manipulation interface.

Problem

• dexterous robot learning
• bare hand video
• tactile supervision
• motion supervision
• human-robot manipulation

Innovation

• Bare hand to exoskeleton mapping
• Human limb coupling
• Tactile supervision
• Dexterous robot learning
• Temporal visual model