VTAP Gripper: Synergizing Fingertip Sensing and a Visuo-Tactile Active Palm for Dexterous In-Hand Manipulation

2026-07-16

AI Summary

This work addresses the challenge of achieving robust grasping and fine manipulation in non-anthropomorphic, high-degree-of-freedom robotic hands by proposing a dexterous gripper that integrates a vision–tactile active palm with soft, reconfigurable fingertips. Through a synergistic finger–palm mechanism and a staged, gesture-conditioned redirection framework, the system eliminates reliance on conventional anthropomorphic architectures. It leverages tactile-array-enabled soft fingers, a bimodal-driven palm, and multimodal sensing to enable efficient in-hand manipulation. Experimental results demonstrate superior performance in tasks including delicate object grasping, syringe alignment and pressing, sorting of 3-mm micro-objects, and vision–tactile-guided peg insertion, significantly enhancing dexterous manipulation capabilities within non-anthropomorphic designs.

Abstract

This paper presents a tactile-reactive gripper that integrates a Visuo-Tactile Active Palm (VTAP) and compliant, reconfigurable fingers equipped with tactile array sensors. The design exploits structured finger-palm synergy and multi-modal perception to achieve both robust grasping and fine manipulation. The actuated bi-modal palm seamlessly combines long-range visual localization with contact-rich tactile feedback, substantially extending the system's manipulation capability. To bridge the embodiment gap between human hand motion and the heterogeneous three-finger structure, we further propose a staged, gesture-conditioned retargeting framework for dexterous teleoperation. Extensive experiments validate the system across a range of challenging tasks: reactive grasping of YCB and fragile objects, in-hand syringe reorientation and plunger actuation, singulation of clustered objects down to 3 mm in diameter, and vision-tactile peg-in-hole insertion. Results demonstrate that high manipulation performance can be achieved through coordinated finger-palm interaction and multi-modal sensing, without resorting to high degrees of freedom anthropomorphic designs. The VTAP gripper and its retargeting framework offer a practical reference architecture for dexterous gripper design, manipulation, and contact-rich data collection in support of learning-based approaches. Project webpage: https://yuhochau.github.io/vtap/.

Problem

• dexterous manipulation
• in-hand manipulation
• tactile sensing
• visuo-tactile perception
• gripper design

Innovation

• visuo-tactile sensing
• active palm
• finger-palm synergy
• dexterous manipulation
• gesture-conditioned retargeting