ReTouch: Empowering Contact-Rich Dexterous Manipulation with Online-Refined Tactile Prediction

2026-08-03

AI Summary

This work addresses the insufficient integration of tactile feedback in contact-rich dexterous manipulation by proposing ReTouch, a vision–language–action model that achieves high-precision and robust manipulation through online refinement of tactile predictions. The method introduces a structured tactile patch encoder to preserve finger identity and local contact geometry, and designs a high-frequency action module that jointly predicts and continuously corrects future tactile states and action sequences in real time. By integrating multimodal tactile–visual–language fusion with closed-loop action generation, ReTouch outperforms the strongest baseline by 18.4% and 23.8% in average success rate under standard and challenging scenarios, respectively, demonstrating significantly enhanced robustness to contact variations and execution errors.

Abstract

Fusing tactile signals has proven effective for contact-rich manipulation, enabling robots to perceive contact states and adapt to rapidly changing physical interactions. Yet effectively integrating tactile feedback into dexterous manipulation remains underexplored. In this work, we introduce ReTouch, a vision-language-action model (VLA) that supports contact-rich dexterous manipulation through tactile predictions continually refined online using execution-time feedback. ReTouch builds on two main innovations for tactile representation and closed-loop action generation. First, its Tactile-Patch Encoder represents tactile observations as structured tactile patch features that preserve finger identity and local contact structure, providing contact cues for fine-grained dexterous control. Second, its high-frequency action module jointly predicts future tactile states and action chunks and refines both using incoming tactile feedback during execution. This closed-loop refinement keeps tactile predictions aligned with evolving physical interactions, enabling responsive action correction and improving robustness to contact changes and execution errors. We further introduce XHT-Dataset, comprising 900 real-world demonstrations across seven contact-rich tasks collected on an XHand--UR7e platform, and evaluate ReTouch through closed-loop real-robot experiments. ReTouch surpasses the strongest baseline by 18.4 and 23.8 percentage points in average success rate under standard and challenging conditions, respectively, demonstrating its effectiveness and robustness.

Problem

• tactile feedback
• dexterous manipulation
• contact-rich tasks
• closed-loop control
• tactile prediction

Innovation

• tactile prediction
• dexterous manipulation
• closed-loop refinement
• tactile-patch encoding
• vision-language-action model