🤖 AI Summary
This work addresses the challenge of coordinating a wheeled mobile base and dual arms during whole-body teleoperation of a mobile dual-arm robot, where operators struggle to simultaneously manage motion and avoid collisions. The authors propose an open-source teleoperation system that maps upper-limb motions to the robot arms while controlling the base via foot pedals. For the first time, lidar-driven haptic feedback is integrated into the foot pedals, complemented by bilateral force reflection and real-time manipulability visualization, enabling natural collision avoidance without explicit obstacle-avoidance controllers. Leveraging a low-cost lidar, multimodal force feedback, and the Action Chunking with Transformers (ACT) policy, the system demonstrates effective coordinated control in long-duration, high-precision tasks on a real dual-arm mobile platform, and shows that teleoperation-derived feedback signals can significantly enhance policy learning performance.
📝 Abstract
Mobile manipulators broaden the operational envelope for robot manipulation. However, the whole-body teleoperation of such robots remains a problem: operators must coordinate a wheeled base and two arms while reasoning about obstacles and contact. Existing interfaces are predominantly hand-centric (e.g., VR controllers and joysticks), leaving foot-operated channels underexplored for continuous base control. We present TriPilot-FF, an open-source whole-body teleoperation system for a custom bimanual mobile manipulator that introduces a foot-operated pedal with lidar-driven pedal haptics, coupled with upper-body bimanual leader-follower teleoperation. Using only a low-cost base-mounted lidar, TriPilot-FF renders a resistive pedal cue from proximity-to-obstacle signals in the commanded direction, shaping operator commands toward collision-averse behaviour without an explicit collision-avoidance controller. The system also supports arm-side force reflection for contact awareness and provides real-time force and visual guidance of bimanual manipulability to prompt mobile base repositioning, thereby improving reach. We demonstrate the capability of TriPilot-FF to effectively ``co-pilot''the human operator over long time-horizons and tasks requiring precise mobile base movement and coordination. Finally, we incorporate teleoperation feedback signals into an Action Chunking with Transformers (ACT) policy and demonstrate improved performance when the additional information is available. We release the pedal device design, full software stack, and conduct extensive real-world evaluations on a bimanual wheeled platform. The project page of TriPilot-FF is http://bit.ly/46H3ZJT.