🤖 AI Summary
To address the challenge of collaborative planning and execution in heterogeneous multi-robot systems for complex, long-horizon tasks, this paper proposes a large language model (LLM)-driven closed-loop framework named Proposal-Execution-Feedback-Adjustment (PEFA). PEFA enables centralized high-level task decomposition coupled with decentralized, dynamic self-reflection and adjustment across heterogeneous actuators—including quadcopters, quadruped robots, and robotic arms—by integrating LLM-based reasoning, multi-agent task allocation, cross-platform action-space alignment, and online feedback-driven optimization. We introduce the first benchmark comprising 100 real-world scenario tasks specifically designed for heterogeneous multi-robot systems. Experimental results demonstrate that our approach achieves significantly higher task success rates and execution efficiency compared to state-of-the-art methods. All code, benchmark datasets, and demonstration videos are publicly released.
📝 Abstract
Leveraging the powerful reasoning capabilities of large language models (LLMs), recent LLM-based robot task planning methods yield promising results. However, they mainly focus on single or multiple homogeneous robots on simple tasks. Practically, complex long-horizon tasks always require collaboration among multiple heterogeneous robots especially with more complex action spaces, which makes these tasks more challenging. To this end, we propose COHERENT, a novel LLM-based task planning framework for collaboration of heterogeneous multi-robot systems including quadrotors, robotic dogs, and robotic arms. Specifically, a Proposal-Execution-Feedback-Adjustment (PEFA) mechanism is designed to decompose and assign actions for individual robots, where a centralized task assigner makes a task planning proposal to decompose the complex task into subtasks, and then assigns subtasks to robot executors. Each robot executor selects a feasible action to implement the assigned subtask and reports self-reflection feedback to the task assigner for plan adjustment. The PEFA loops until the task is completed. Moreover, we create a challenging heterogeneous multi-robot task planning benchmark encompassing 100 complex long-horizon tasks. The experimental results show that our work surpasses the previous methods by a large margin in terms of success rate and execution efficiency. The experimental videos, code, and benchmark are released at https://github.com/MrKeee/COHERENT.