StereoTacTip: Vision-based Tactile Sensing with Biomimetic Skin-Marker Arrangements

2025-06-22

AI Summary

Conventional bionic vision-based tactile sensors (VBTSs) suffer from low accuracy and poor robustness in skin-surface geometric reconstruction due to complex, densely packed marker arrangements. Method: This paper proposes StereoTacTip—a stereo-vision-based system for high-fidelity 3D topography sensing. It introduces a Delaunay-triangulation ring coding algorithm for robust marker matching and tracking; establishes a refraction-depth correction model to compensate for optical distortion; and devises a surface-normal inversion method for geometric refinement of the skin surface, enabling precise reconstruction under both single-point and multi-contact conditions. Contribution/Results: Experiments demonstrate substantial improvements in reconstruction accuracy and stability across large-scale 3D topographic mapping. StereoTacTip provides a scalable, geometry-aware sensing paradigm for bioinspired tactile perception, advancing the state of the art in vision-based tactile sensing.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Vision-Based Tactile Sensors (VBTSs) stand out for their superior performance due to their high-information content output. Recently, marker-based VBTSs have been shown to give accurate geometry reconstruction when using stereo cameras. uhl{However, many marker-based VBTSs use complex biomimetic skin-marker arrangements, which presents issues for the geometric reconstruction of the skin surface from the markers}. Here we investigate how the marker-based skin morphology affects stereo vision-based tactile sensing, using a novel VBTS called the StereoTacTip. To achieve accurate geometry reconstruction, we introduce: (i) stereo marker matching and tracking using a novel Delaunay-Triangulation-Ring-Coding algorithm; (ii) a refractive depth correction model that corrects the depth distortion caused by refraction in the internal media; (iii) a skin surface correction model from the marker positions, relying on an inverse calculation of normals to the skin surface; and (iv)~methods for geometry reconstruction over multiple contacts. To demonstrate these findings, we reconstruct topographic terrains on a large 3D map. Even though contributions (i) and (ii) were developed for biomimetic markers, they should improve the performance of all marker-based VBTSs. Overall, this work illustrates that a thorough understanding and evaluation of the morphologically-complex skin and marker-based tactile sensor principles are crucial for obtaining accurate geometric information.

Problem

• Addressing geometric reconstruction challenges in marker-based VBTSs with complex skin-marker arrangements
• Correcting depth distortion caused by refraction in internal media for accurate sensing
• Improving skin surface reconstruction from markers via inverse normal calculation

Innovation

• Delaunay-Triangulation-Ring-Coding for stereo marker tracking
• Refractive depth correction model for distortion
• Inverse calculation of normals for skin surface