Advances in Compliance Detection: Novel Models Using Vision-Based Tactile Sensors

2025-06-17

AI Summary

Traditional compliance measurement devices are costly and non-scalable, while existing vision-based tactile methods suffer from insufficient accuracy. Method: This work proposes a portable, real-time compliance estimation framework for robotic applications. Leveraging RGB tactile images captured by a GelSight sensor, we introduce the first end-to-end prediction model that integrates Long Short-Term Memory Convolutional Networks (LRCN) with Transformer architectures. Crucially, we identify and characterize the relative hardness—i.e., the ratio of object stiffness to sensor stiffness—as the key factor governing estimation difficulty. Results: Extensive experiments demonstrate that our method significantly outperforms baseline models across multiple evaluation metrics, achieving high-accuracy compliance parameter estimation. Notably, we empirically validate the degradation in prediction performance when object stiffness exceeds sensor stiffness—a finding that advances fundamental understanding of visuo-tactile perception and provides a practical tool for robotic tactile sensing.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Compliance is a critical parameter for describing objects in engineering, agriculture, and biomedical applications. Traditional compliance detection methods are limited by their lack of portability and scalability, rely on specialized, often expensive equipment, and are unsuitable for robotic applications. Moreover, existing neural network-based approaches using vision-based tactile sensors still suffer from insufficient prediction accuracy. In this paper, we propose two models based on Long-term Recurrent Convolutional Networks (LRCNs) and Transformer architectures that leverage RGB tactile images and other information captured by the vision-based sensor GelSight to predict compliance metrics accurately. We validate the performance of these models using multiple metrics and demonstrate their effectiveness in accurately estimating compliance. The proposed models exhibit significant performance improvement over the baseline. Additionally, we investigated the correlation between sensor compliance and object compliance estimation, which revealed that objects that are harder than the sensor are more challenging to estimate.

Problem

• Improving compliance detection accuracy using vision-based tactile sensors
• Overcoming portability and scalability limits in traditional compliance methods
• Addressing challenges in estimating compliance for harder objects

Innovation

• LRCN and Transformer models for compliance detection
• Uses RGB tactile images from GelSight sensor
• Improves accuracy over traditional neural networks