MVP-Tac: A Miniaturized Dual-Modal Vision and Photoelastic Tactile Sensor for Robot-Assisted Minimally Invasive Surgery

2026-07-20

AI Summary

This work addresses the longstanding challenge in minimally invasive surgery where visual and tactile sensing are mutually exclusive, hindering safe navigation and effective palpation. The authors propose MVP-Tac, a compact dual-modality sensor that integrates visual imaging and reflection-based photoelastic tactile perception within a single device through a switchable semi-transparent membrane and controllable illumination, achieving the first miniaturized co-located and concurrent visual-tactile design. The system incorporates an embedded polarizer, a photoelastic elastomer, and video analysis algorithms to enable seamless mode switching. Calibrated over a 0–2 N force range, it achieves 97% and 92% accuracy in classifying the stiffness of exposed and subcutaneous tumor models, respectively, and successfully demonstrates intraluminal 3D visual mapping with in situ nodule hardness identification in a simulated colonoscopy. All hardware and software components are open-sourced.

Abstract

Robot-assisted minimally invasive surgery (RMIS) offers major benefits over open and conventional laparoscopic procedures, yet it still lacks tactile feedback for palpation while operating under strict requirements to preserve reliable vision for navigation and safety. In practice, visual feedback is indispensable, and tactile solutions that cannot coexist with vision are difficult to translate into RMIS tools. To address both needs, we introduce MVP-Tac, a compact, vision-based tactile sensor that provides co-located vision and tactile sensing. MVP-Tac uses reflective photoelastic imaging: a thin photoelastic elastomer produces stress-dependent interferograms under contact that are captured by an embedded camera through a miniaturized reflective polariscope. A semi-transparent membrane and controllable illumination enable switching between visual mode and tactile mode, enabling tactile perception without sacrificing vision. We validate MVP-Tac through force calibration in the 0 to 2 N range and demonstrate its potential for tumor palpation via video-based hardness classification on tissue phantoms, achieving 97% accuracy for exposed-tumor classification and 92% accuracy for subdermal-tumor classification. Finally, we conduct a simulated colonoscopy to validate both visual and tactile modalities in a constrained lumen, including vision-guided 3D photomapping of the luminal wall and in situ hardness classification of localized nodules. Overall, MVP-Tac provides a practical path toward restoring clinically useful palpation in RMIS while maintaining essential visual feedback. The design, fabrication, and firmware of MVP-Tac are open-sourced at https://mvp-tac.github.io/

Problem

• robot-assisted minimally invasive surgery
• tactile feedback
• vision-tactile coexistence
• palpation
• surgical robotics

Innovation

• dual-modal sensing
• photoelastic tactile sensor
• miniaturized endoscopic sensor
• vision-tactile co-location
• robot-assisted minimally invasive surgery