TactiVerse: Generalizing Multi-Point Tactile Sensing in Soft Robotics Using Single-Point Data

2026-02-23

AI Summary

This work addresses the challenge of limited generalization in real-time deformation prediction for highly compliant materials in soft robotics, particularly under complex tactile scenarios such as multi-point contact. The authors formulate contact geometry estimation as a spatial heatmap prediction task using a U-Net architecture, which—trained solely on single-indentation data—effectively generalizes to multi-point and large-area contact deformation sensing, thereby overcoming the strong dependence on training data distribution inherent in conventional models. Integrating marker-based visual soft tactile sensing with tailored data augmentation strategies, the method achieves a mean absolute error (MAE) of 0.0589 mm in single-point indentation estimation, outperforming baseline approaches. Notably, it reduces the MAE for two-point contact localization from 1.214 mm to 0.383 mm, substantially enhancing multi-point tactile perception performance.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Real-time prediction of deformation in highly compliant soft materials remains a significant challenge in soft robotics. While vision-based soft tactile sensors can track internal marker displacements, learning-based models for 3D contact estimation heavily depend on their training datasets, inherently limiting their ability to generalize to complex scenarios such as multi-point sensing. To address this limitation, we introduce TactiVerse, a U-Net-based framework that formulates contact geometry estimation as a spatial heatmap prediction task. Even when trained exclusively on a limited dataset of single-point indentations, our architecture achieves highly accurate single-point sensing, yielding a superior mean absolute error of 0.0589 mm compared to the 0.0612 mm of a conventional regression-based CNN baseline. Furthermore, we demonstrate that augmenting the training dataset with multi-point contact data substantially enhances the sensor's multi-point sensing capabilities, significantly improving the overall mean MAE for two-point discrimination from 1.214 mm to 0.383 mm. By successfully extrapolating complex contact geometries from fundamental interactions, this methodology unlocks advanced multi-point and large-area shape sensing. Ultimately, it significantly streamlines the development of marker-based soft sensors, offering a highly scalable solution for real-world tactile perception.

Problem

• soft robotics
• tactile sensing
• multi-point contact
• generalization
• deformation prediction

Innovation

• TactiVerse
• soft tactile sensing
• multi-point contact
• spatial heatmap prediction
• U-Net