FasTac: A Curved Multispectral Vision-Based Tactile Sensor for High-Speed High-Precision 3D Shape and Force Perception

2026-07-30

AI Summary

This work addresses the challenge of simultaneously achieving high-precision 3D surface reconstruction, triaxial force estimation, and real-time performance in compact curved visual tactile sensors. The authors propose an end-to-end perception framework that integrates multispectral photometric stereo, boundary-prior Poisson depth reconstruction, and position-aware dynamic convolution (HyperForce), with FPGA-based hardware acceleration. By leveraging a single image sensor for synchronized multispectral imaging, the method significantly enhances both geometric and mechanical sensing capabilities: it achieves a depth reconstruction mean absolute error (MAE) of 0.0415 mm, normalized mean absolute errors (NMAE) of 2.74% and 2.39% for normal and tangential forces, respectively, and reduces processing latency from 3.26 ms to 1.09 ms. This represents the first demonstration of sub-millimeter accuracy and millisecond-level response in curved tactile sensing, enabling multi-object reconstruction, feedback-based grasping, and vibration measurement.

Abstract

Curved tactile fingertips for dexterous manipulation must resolve fine contact geometry, distinguish normal and tangential loads, and capture transient signals. Existing curved vision-based tactile sensors struggle to combine accurate 3D reconstruction, three-axis force estimation, and high-speed processing in a compact form. This article presents FasTac, a curved vision-based tactile sensor integrating multispectral photometric stereo, dynamic-convolution force estimation, and hardware acceleration on a field-programmable gate array (FPGA). Single-image-sensor simultaneous multispectral imaging provides spatially aligned observations for robust surface normal estimation, followed by boundary-prior fast Poisson depth reconstruction. HyperForce uses position-aware dynamic convolution to model the spatially nonuniform mechanical response of curved elastomers and estimate three-axis forces. The complete image-to-normal-force pipeline is deployed on an FPGA. Experiments show that near-infrared (NIR) illumination and the boundary prior decrease depth mean absolute error (MAE) from 0.2730 mm to 0.0415 mm; HyperForce achieves normalized mean absolute error (NMAE) values of 2.74% and 2.39% for normal and shear forces, respectively; and FPGA deployment shortens processing latency from 3.26 ms on the GPU to 1.09 ms. Multi-object reconstruction, feedback grasping, and vibration measurement validate fine geometric perception, stable force feedback, and dynamic contact sensing.

Problem

• curved tactile sensor
• 3D shape reconstruction
• three-axis force estimation
• high-speed processing
• multispectral vision

Innovation

• multispectral photometric stereo
• dynamic-convolution force estimation
• FPGA acceleration
• curved tactile sensor
• 3D shape reconstruction