WT-UMI: Tactile-based Whole-Body Manipulation via Force-Supervised Contact-Aware Planning

2026-06-11

AI Summary

This work addresses the lack of distributed tactile perception and explicit force regulation in humanoid robots when manipulating heavy, deformable, or co-carried objects. To this end, the authors propose a force-supervised contact-aware motion planning framework that integrates human demonstrations and teleoperation data. A wearable full-body tactile interface captures tactile images, contact forces, and end-effector poses, which are fed into a force-supervised trajectory prediction module and a force-conditioned goal pose refinement module to generate contact-aware robot motion commands. These commands are executed using tactile admittance control to achieve high-fidelity force interaction. Experimental results across five tasks with high contact complexity demonstrate that the proposed method significantly outperforms four baseline strategies, substantially improving task success rates and reducing contact position tracking errors.

Abstract

Whole-body humanoid manipulation of bulky, deformable, and shared-load objects requires distributed contact sensing and explicit force regulation, yet most imitation policies treat contact force only implicitly. On the other hand, different demonstration sources provide complementary modalities with inherent trade-offs: human demonstrations capture natural contact forces but not robot-executable actions, while teleoperation directly records robot actions but with less natural force regulation. This paper presents \textbf{WT-UMI}, a wearable whole-body tactile interface worn by human operators or mounted on humanoids, providing accurate observations of tactile images, contact forces, and end-effector poses across both human demonstration and humanoid teleoperation modes. We introduce a force-conditioned target-pose correction module that converts measured human poses into contact-aware robot targets by learning corrections from teleoperation data. To leverage the natural force interaction in human data, we propose a force-supervised planner that predicts end-effector pose chunks and contact-force trajectories. The predicted contact force serves as the reference for a tactile-based admittance controller. Across five contact-rich tasks spanning deformable objects, bulky rigid objects, and human--humanoid collaboration, WT-UMI improves success rate and reduces contact-position tracking error over four policy baselines. Our project page is available at https://wt-umi.github.io/WTUMI/.

Problem

• whole-body manipulation
• contact-aware planning
• force regulation
• tactile sensing
• humanoid robotics

Innovation

• tactile sensing
• force-supervised planning
• whole-body manipulation
• contact-aware control
• humanoid teleoperation