Tactile Reconstruction of Contact Task Frames and Forces for Hybrid Force/Motion Control

2026-10-07

AI Summary

This study addresses the significant challenge in hybrid force/motion control of relying exclusively on soft tactile sensing to online estimate contact forces and time-varying task frames. To overcome this, the work proposes an end-to-end mapping model based on single-frame optical tactile images, coupled with a self-annotated data acquisition pipeline, enabling real-time reconstruction of contact variables without requiring a nominal environment model. Furthermore, state estimation is achieved by integrating an extended Kalman filter (EKF) with robot proprioceptive data. The proposed approach is validated on a UR10 manipulator, demonstrating closed-loop contact force regulation under both linear and angular motions. These results confirm that high-precision hybrid control can be realized solely through tactile feedback, eliminating the need for external sensors.

Abstract

Hybrid force/motion control requires knowledge of the interaction force and of a task frame defining the force- and motion-controlled directions. These quantities are usually obtained from force/torque sensing or model-based residuals, often assuming also a nominal environment model. This work addresses the online estimation of the contact force and a possibly time-varying task frame using only soft optical tactile sensing, under the assumption of locally planar contact with a negligible contact moment. The proposed method maps a single image of the deformed elastomer of a soft optical tactile sensor to observable contact variables: indentation depth, two surface-to-sensor tilt angles, and 3D contact force, each with a per-sample uncertainty estimate. The mapping is learned through a self-labeling acquisition procedure, in which a manipulator imposes controlled contacts while an auxiliary Force/Torque sensor is used offline to provide ground-truth labels. The tactile measurement is then fused with robot proprioceptive data in an Extended Kalman Filter, producing a continuously updated estimate of the contact task frame and of the interaction force. Control experiments with a DigiTac sensor mounted on a UR10 manipulator demonstrate closed-loop contact force regulation against a flat rigid board in linear and angular motion by a human operator, with touch as the only exteroceptive feedback.

Problem

• hybrid force/motion control
• tactile sensing
• contact force estimation
• task frame
• online estimation

Innovation

• soft optical tactile sensing
• hybrid force/motion control
• task frame estimation
• self-labeling acquisition
• Extended Kalman Filter