Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning

2026-06-10

AI Summary

This work addresses the challenge of enabling blind grasping with real multi-fingered dexterous hands using only tactile sensing, without relying on vision or real-world demonstrations. The authors propose a vision-free and demonstration-free approach that leverages Real2Sim tactile calibration to construct a high-fidelity digital twin simulator. They design a layout-aware tactile encoder and enhance the representation of sparse tactile signals through geometric prior-guided self-supervised learning. Object-specific policies trained in simulation are unified via a tactile-conditioned diffusion model into a single generalist policy. Evaluated on the LEAP hand across 20 objects—including 10 previously unseen—the method achieves a 27% success rate in blind grasping, demonstrating significantly improved generalization in real-world settings.

Abstract

Blind grasping with a dexterous hand is a crucial manipulation capability. Nevertheless, learning such tactile-only policies for real robots remains challenging due to the tactile sim-to-real gap and the limited expressiveness of sparse tactile signals. To bridge this gap, we propose a framework for tactile-only blind grasping that is deployable on a physical multi-fingered robotic hand. Our approach combines three key components. First, we introduce a Real2Sim tactile calibration pipeline that constructs a contact-calibrated digital-twin simulator capable of reproducing real tactile signals. Second, we improve the expressiveness of sparse tactile observations using a layout-aware tactile encoder, which incorporates sensor-geometry priors through self-supervised pretraining. Third, to improve generalization to unseen objects, we train object-specific reinforcement-learning experts in the calibrated simulator and aggregate their successful grasp trajectories into a tactile-conditioned Diffusion Policy. We evaluate our method on a physical LEAP Hand equipped with distributed tactile sensing across 10 seen and 10 unseen objects. The deployed policy achieves a 27\% real-world grasp success rate across all 20 objects, without real-world grasping demonstrations or visual input. Simulation ablations show that layout-aware tactile pretraining improves grasping performance, while sensing-level evaluations confirm that Real2Sim calibration increases the consistency of tactile contact events between simulation and hardware. Together, these results suggest that contact-event calibration, geometry-aware tactile representation learning, and diffusion-based policy aggregation provide an effective path toward tactile-only blind grasping on real dexterous robotic hands. Project page:Dex-Blind-Grasp.github.io.

Problem

• blind grasping
• dexterous hand
• tactile-only policy
• sim-to-real gap
• sparse tactile signals

Innovation

• Real2Sim2Real
• tactile policy learning
• layout-aware tactile encoder
• Diffusion Policy
• digital-twin simulator