Self-Supervised Anchoring of Fingertip Sensing to Proprioception and Proactive Actions for Robot Imitation Learning

2026-09-24

AI Summary

This study addresses the challenge that fingertip sensing is susceptible to occlusion and difficult to integrate with vision-based policies, resulting in the underutilization of local interaction cues in imitation learning. To this end, this work proposes PROPRA, a framework that independently anchors fingertip pressure and proximity signals to proprioception and active actions for staged alignment. Furthermore, it introduces a novel proprioceptive-action self-supervised pretraining mechanism that effectively overcomes the difficulty of extracting sparse, phase-dependent sensor signals from limited demonstrations. Empirical evaluations on real-world manipulation tasks demonstrate that PROPRA significantly outperforms purely visual and image-anchored baselines, substantially improving success rates while preserving richer pre-contact state information.

Abstract

Robotic imitation learning often relies on external cameras, yet local interaction cues such as object proximity, contact onset, and grasp state are difficult to observe near the fingertips because of occlusion and limited temporal resolution. We study how to effectively incorporate complementary fingertip sensing into imitation learning using pressure-sensitive tactile and reflective proximity sensors, along with pretrained sensor encoders. The two modalities provide information at different manipulation phases: proximity sensing is informative before contact, whereas tactile sensing becomes informative after contact. However, naively adding these signals to a policy does not consistently improve performance and can even underperform vision-only policies, suggesting that sparse, phase-dependent sensor signals are difficult to exploit from limited demonstrations. We therefore propose a proprioception-anchored pretraining method, PROprioceptive-and-PRoactive Anchoring (PROPRA), which independently aligns each fingertip sensor history with proprioceptive and action segments. This provides a continuously available sensorimotor reference, allowing each sensor to be aligned independently during its informative phases. Experiments on real-world manipulation tasks show that our pretraining method improves average success rates over vision-only policies and image-anchored pretraining baselines. Representation analysis further shows that it preserves richer information about pre-contact states, enabling more effective use of complementary fingertip sensing. Please refer to our project page: https://tomohiromotoda.github.io/nia.propra/

Problem

• imitation learning
• fingertip sensing
• tactile sensing
• proximity sensing
• proprioception

Innovation

• Imitation Learning
• Self-Supervised Pretraining
• Fingertip Sensing
• Proprioception Anchoring
• Tactile and Proximity Sensors