🤖 AI Summary
High-fidelity tactile modeling conflicts with real-time performance, and existing methods rely heavily on scarce real-world labeled data. Method: This paper introduces the first hydroelastic contact mechanics–based tactile sensor simulation framework, enabling continuous pressure-field modeling and sensor signal synthesis for soft–soft and soft–hard non-convex surface interactions. Implemented as an efficient, plugin-based extension in MuJoCo, it ensures physical fidelity via pressure-surface discretized integration while maintaining computational efficiency. Contribution/Results: The method achieves zero-shot sim-to-real transfer—using only synthetic data, it significantly improves real-sensor performance in object state estimation. It overcomes the long-standing accuracy–speed trade-off inherent in point-contact and finite-element approaches. The code is open-sourced and integrated into the MuJoCo ecosystem.
📝 Abstract
Thanks to recent advancements in the development of inexpensive, high-resolution tactile sensors, touch sensing has become popular in contact-rich robotic manipulation tasks. With the surge of data-driven methods and their requirement for substantial datasets, several methods of simulating tactile sensors have emerged in the tactile research community to overcome real-world data collection limitations. These simulation approaches can be split into two main categories: fast but inaccurate (soft) point-contact models and slow but accurate finite element modeling. In this work, we present a novel approach to simulating pressure-based tactile sensors using the hydroelastic contact model, which provides a high degree of physical realism at a reasonable computational cost. This model produces smooth contact forces for soft-to-soft and soft-to-rigid contacts along even non-convex contact surfaces. Pressure values are approximated at each point of the contact surface and can be integrated to calculate sensor outputs. We validate our models' capacity to synthesize real-world tactile data by conducting zero-shot sim-to-real transfer of a model for object state estimation. Our simulation is available as a plug-in to our open-source, MuJoCo-based simulator.