RoboPilot: Generalizable Dynamic Robotic Manipulation with Dual-thinking Modes

2025-09-30

AI Summary

Autonomous robots exhibit poor robustness and severe error accumulation in complex, long-horizon tasks. Method: This paper proposes RoboPilot, a dual-mode closed-loop operational framework integrating “fast thinking” (real-time perception–execution) with “slow thinking” (chain-of-reasoning-driven dynamic replanning) to enable environment-adaptive decision-making and execution feedback closure. Planning is structured via primitive actions, unifying high-level reasoning and low-level control. Contribution/Results: We introduce RoboPilot-Bench, a benchmark comprising 21 diverse tasks—the first to support infeasible-task identification and failure-recovery validation. Experiments show a 25.9% absolute improvement in task success rate over state-of-the-art methods. RoboPilot is deployed on an industrial robotic platform, demonstrating enhanced operational robustness and generalization in dynamic real-world environments.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

Despite rapid progress in autonomous robotics, executing complex or long-horizon tasks remains a fundamental challenge. Most current approaches follow an open-loop paradigm with limited reasoning and no feedback, resulting in poor robustness to environmental changes and severe error accumulation. We present RoboPilot, a dual-thinking closed-loop framework for robotic manipulation that supports adaptive reasoning for complex tasks in real-world dynamic environments. RoboPilot leverages primitive actions for structured task planning and flexible action generation, while introducing feedback to enable replanning from dynamic changes and execution errors. Chain-of-Thought reasoning further enhances high-level task planning and guides low-level action generation. The system dynamically switches between fast and slow thinking to balance efficiency and accuracy. To systematically evaluate the robustness of RoboPilot in diverse robot manipulation scenarios, we introduce RoboPilot-Bench, a benchmark spanning 21 tasks across 10 categories, including infeasible-task recognition and failure recovery. Experiments show that RoboPilot outperforms state-of-the-art baselines by 25.9% in task success rate, and the real-world deployment on an industrial robot further demonstrates its robustness in real-world settings.

Problem

• Executing complex robotic tasks with limited reasoning capabilities
• Addressing poor robustness to environmental changes and errors
• Enabling adaptive manipulation in dynamic real-world environments

Innovation

• Dual-thinking closed-loop framework for robotic manipulation
• Primitive actions with feedback for replanning and error recovery
• Dynamic switching between fast and slow thinking modes