TouchSight: Bare-Handed Tactile Prediction from Egocentric Video via Generative Visual Augmentation

2026-09-17

AI Summary

为解决触觉信号采集困难问题，本文提出TouchSight框架，利用500小时的压力手套记录和手-物体交互数据，通过生成视觉增强技术预测裸手触觉。

Abstract

Tactile signals provide direct contact and force measurements that are essential for understanding physical interactions and enabling dexterous robotic manipulation. However, tactile sensing requires direct measurement at contact interfaces, making large-scale data collection reliant on intrusive, costly, and restrictive instrumentation. We present TouchSight, a monocular egocentric vision framework for dense full-hand contact force prediction that leverages 500 hours of pressure-glove recordings and extensive hand-object interaction (HOI) data. To address the appearance gap between gloved training data and bare-hand real-world scenarios, we construct TwinTouch-20H: 20 hours of paired visual data in which generative video models re-render gloved recordings as bare-hand observations against new backgrounds while preserving the original measured tactile labels. TouchSight predicts dense force from both gloved and generated bare-hand videos, outperforms prior contact prediction methods on OakInk2, qualitatively generalizes to natural bare-hand egocentric videos from unseen datasets, and improves consistently as glove supervision scales. These results demonstrate that dense tactile signals can be recovered from egocentric vision alone, without tactile instrumentation at capture time.

Problem

• tactile prediction
• egocentric video
• contact force
• vision-based

Innovation

• Egocentric Vision
• Tactile Prediction
• Generative Visual Augmentation
• Dense Contact Force