Near-sensor Computing for Rapid Visuotactile Perception

2026-08-06

AI Summary

This work addresses the limitations of conventional host-based visual-tactile processing—namely high power consumption, substantial transmission latency, and non-deterministic scheduling—which hinder robotic perception and response performance. The authors propose a near-sensor computing framework that enables deterministic, low-latency tactile reconstruction directly at the sensor without relying on data-dependent branching or iterative convergence. The architecture integrates a fully pipelined hardware design, a spectral Poisson solver, fixed-point arithmetic, and on-chip decision logic, operating at 166 MHz. Experimental results demonstrate a first-pixel latency of only 0.211 ms per frame, a reconstruction error of 0.17% relative to peak depth, a protective reflex closed-loop latency of 28.3 ± 4.9 ms, and a power consumption of merely 347 mW.

Abstract

Visuotactile sensors reconstruct dense contact geometry from measured surface gradients, but host-based processing increases power consumption and introduces data-transfer delays and variable scheduling latency, limiting the sensing and response speed of robotic systems. To address these limitations, we implement a near-sensor computing framework that includes a spectral Poisson solver as a fully streaming hardware pipeline. The computational core logic has an estimated power consumption of 347 mW and achieves high throughput without data-dependent branching or iterative convergence, thereby providing deterministic latency. Operating at 166 MHz, the pipeline produces the first depth value of each 128x128 frame 35,107 cycles after receiving the first input pixel, corresponding to a fixed latency of 0.211 ms. Across 15 contact geometries, the reconstructed depths differ from a double-precision reference by 0.17 % of the peak contact depth. On-chip decisions based on these reconstructions close a robot protective reflex loop in 28.3 +/- 4.9 ms, compared with 169.9 +/- 27.8 ms for an equivalent host-based loop using the same actuator. These results demonstrate that near-sensor reconstruction can provide accurate, energy-efficient, and deterministic tactile geometry on timescales suitable for rapid robotic contact responses.

Problem

• visuotactile perception
• near-sensor computing
• data-transfer delay
• scheduling latency
• robotic response speed

Innovation

• near-sensor computing
• visuotactile perception
• spectral Poisson solver
• deterministic latency
• hardware pipeline