Tacchi 2.0: A Low Computational Cost and Comprehensive Dynamic Contact Simulator for Vision-based Tactile Sensors

2025-03-12

AI Summary

To address the high information acquisition cost of vision-tactile sensors in contact-rich tasks and the trade-off between simulation robustness and efficiency, this paper introduces Tacchi 2.0—the first lightweight Material Point Method (MPM)-based dynamic tactile simulator integrating a pinhole camera model. It enables joint, high-fidelity generation of tactile images, marker motion images, and joint images under diverse contact modalities—including pressing, sliding, and rotating. Crucially, it embeds geometric imaging modeling directly into the physics simulation framework, reducing computational overhead by one to two orders of magnitude compared to finite element methods while enhancing cross-sensor generalizability. Experimental results demonstrate that Tacchi 2.0’s synthetic data achieve high fidelity and strong robustness across multiple vision-tactile hardware platforms, outperforming purely data-driven approaches in both accuracy and adaptability.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Web Mining and Content Analysis: Web data generation and simulation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

With the development of robotics technology, some tactile sensors, such as vision-based sensors, have been applied to contact-rich robotics tasks. However, the durability of vision-based tactile sensors significantly increases the cost of tactile information acquisition. Utilizing simulation to generate tactile data has emerged as a reliable approach to address this issue. While data-driven methods for tactile data generation lack robustness, finite element methods (FEM) based approaches require significant computational costs. To address these issues, we integrated a pinhole camera model into the low computational cost vision-based tactile simulator Tacchi that used the Material Point Method (MPM) as the simulated method, completing the simulation of marker motion images. We upgraded Tacchi and introduced Tacchi 2.0. This simulator can simulate tactile images, marked motion images, and joint images under different motion states like pressing, slipping, and rotating. Experimental results demonstrate the reliability of our method and its robustness across various vision-based tactile sensors.

Problem

• High computational cost of tactile data simulation
• Lack of robustness in data-driven tactile data generation
• Durability issues increasing tactile information acquisition cost

Innovation

• Integrates pinhole camera model for tactile simulation
• Uses Material Point Method for low computational cost
• Simulates tactile, marker, and joint images effectively