Norm2Tex: Augmenting Visuo-Tactile Simulations with Texture

2026-09-21

AI Summary

为解决触觉模拟与真实数据间因缺乏纹理细节造成的差异问题，本文提出Norm2Tex方法，通过增强视觉触觉传感器模拟中的表面细节来改进仿真效果。

Abstract

Large-scale datasets are essential for training generalist robot control policies. Collecting real-world tactile data is costly and time-consuming, motivating the use of tactile simulations. However, current tactile simulators capture only overall contact geometry and miss fine details like texture. This results in a significant domain shift between simulated and real tactile data. To address this gap, we introduce Norm2Tex, a plug-in method that augments simulations of vision-based tactile sensors with high-frequency surface details from normal map textures. By modifying the target object's depth map before a tactile simulator's rendering pipeline, Norm2Tex seamlessly integrates into different tactile simulators. We also evaluate sim-to-real transfer using material classification and a reinforcement learning task. Our results show that Norm2Tex preserves material-dependent tactile information across domains, improving texture recognition and producing material-dependent control behavior in the real world.

Problem

• tactile simulations
• texture
• domain shift

Innovation

• Norm2Tex
• tactile simulations
• normal map textures
• material-dependent tactile information
• sim-to-real transfer