Heavy lifting tasks via haptic teleoperation of a wheeled humanoid

2025-05-26

AI Summary

Wheeled humanoid robots face significant challenges in dynamic mobile manipulation (DMM) due to difficulties in coordinating locomotion, manipulation, and posture control under high interactive loads. Method: This paper proposes a tactile teleoperation framework integrating whole-body motion retargeting with real-time force feedback. It combines human motion capture, endpoint 6-DoF wrench mapping, balance-aware visual/audio cues, and tilt-assisted stabilization, enabling hybrid human-in-the-loop and autonomous balance regulation. Contribution/Results: To our knowledge, this is the first approach achieving full-body haptic closed-loop control for wheeled humanoids under dynamic loading. Experiments demonstrate stable dynamic lifting of barbells and boxes up to 2.5 kg—21% of the robot’s total mass—exhibiting strong adaptability, disturbance robustness, and whole-body coordination. The framework overcomes critical technical bottlenecks in heavy-load DMM for wheeled humanoids.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Interaction Techniques and Devices
• Computer Vision: Motion & Tracking

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Humanoid robots can support human workers in physically demanding environments by performing tasks that require whole-body coordination, such as lifting and transporting heavy objects.These tasks, which we refer to as Dynamic Mobile Manipulation (DMM), require the simultaneous control of locomotion, manipulation, and posture under dynamic interaction forces. This paper presents a teleoperation framework for DMM on a height-adjustable wheeled humanoid robot for carrying heavy payloads. A Human-Machine Interface (HMI) enables whole-body motion retargeting from the human pilot to the robot by capturing the motion of the human and applying haptic feedback. The pilot uses body motion to regulate robot posture and locomotion, while arm movements guide manipulation.Real time haptic feedback delivers end effector wrenches and balance related cues, closing the loop between human perception and robot environment interaction. We evaluate the different telelocomotion mappings that offer varying levels of balance assistance, allowing the pilot to either manually or automatically regulate the robot's lean in response to payload-induced disturbances. The system is validated in experiments involving dynamic lifting of barbells and boxes up to 2.5 kg (21% of robot mass), demonstrating coordinated whole-body control, height variation, and disturbance handling under pilot guidance. Video demo can be found at: https://youtu.be/jF270_bG1h8?feature=shared

Problem

• Teleoperation framework for heavy lifting tasks using wheeled humanoid robots
• Whole-body motion retargeting with haptic feedback for human-robot coordination
• Dynamic mobile manipulation under payload-induced disturbances and balance control

Innovation

• Teleoperation framework for wheeled humanoid robots
• Whole-body motion retargeting with haptic feedback
• Real-time balance assistance during heavy lifting