MLM: Learning Multi-task Loco-Manipulation Whole-Body Control for Quadruped Robot with Arm

2025-08-14

AI Summary

This work addresses the challenge of whole-body coordinated locomotion and manipulation control for quadrupedal robots equipped with robotic arms. We propose a reinforcement learning framework that integrates simulation and real-world data. Our method introduces two key innovations: (1) a trajectory library mechanism with adaptive curriculum sampling to enhance cross-task generalization; and (2) a trajectory-velocity prediction policy network that jointly models future states and enables zero-shot transfer. The framework supports both autonomous execution and teleoperation modes. A systematic ablation study is conducted in simulation, and multi-task zero-shot transfer is successfully demonstrated on a physical platform (Unitree Go2 quadruped + Airbot Arm). Experimental results show significant improvements in whole-body locomotion-manipulation coupling accuracy and task adaptability. This approach provides a scalable solution for multi-scale manipulation control in embodied agents.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Whole-body loco-manipulation for quadruped robots with arm remains a challenging problem, particularly in achieving multi-task control. To address this, we propose MLM, a reinforcement learning framework driven by both real-world and simulation data. It enables a six-DoF robotic arm--equipped quadruped robot to perform whole-body loco-manipulation for multiple tasks autonomously or under human teleoperation. To address the problem of balancing multiple tasks during the learning of loco-manipulation, we introduce a trajectory library with an adaptive, curriculum-based sampling mechanism. This approach allows the policy to efficiently leverage real-world collected trajectories for learning multi-task loco-manipulation. To address deployment scenarios with only historical observations and to enhance the performance of policy execution across tasks with different spatial ranges, we propose a Trajectory-Velocity Prediction policy network. It predicts unobservable future trajectories and velocities. By leveraging extensive simulation data and curriculum-based rewards, our controller achieves whole-body behaviors in simulation and zero-shot transfer to real-world deployment. Ablation studies in simulation verify the necessity and effectiveness of our approach, while real-world experiments on the Go2 robot with an Airbot robotic arm demonstrate the policy's good performance in multi-task execution.

Problem

• Achieving multi-task whole-body loco-manipulation for quadruped robots with arm
• Balancing multiple tasks during loco-manipulation learning with adaptive sampling
• Enhancing policy execution across tasks with different spatial ranges

Innovation

• Reinforcement learning with real and simulation data
• Trajectory library with adaptive sampling mechanism
• Trajectory-Velocity Prediction policy network