Recova: Agent-Guided Failure Recovery for Autonomous Robotic Manipulation

2026-10-01

AI Summary

This study addresses the challenge of autonomous recovery from manipulation failures in robotics, where learning corrective behaviors typically requires extensive failure exploration and physical priors. To overcome this bottleneck, we propose an agent-guided digital twin framework that jointly trains task execution and recovery policies in simulation. By integrating agent-based fault diagnosis, procedural testing, and multi-workstation parallel DAgger imitation learning, the method transforms failure experiences into reusable skills, enabling closed-loop sim-to-real iteration and human-machine collaborative capability expansion. Experimental evaluations demonstrate that the proposed framework achieves success rates of 78.8% and 64.9% on the LIBERO-Pro and MolmoSpaces benchmarks, respectively. Furthermore, real-world deployment yields an improved success rate of 87.5% while reducing human intervention to 0%.

Abstract

Manipulation failures can leave scenes in states from which a task policy cannot recover. Learning corrective behaviors requires scalable failure exploration and physical grounding. We present Recova, an agent-guided framework that jointly develops task execution and recovery in a reconstructed digital twin, then verifies and refines both through real-world experience. In the twin, the agent diagnoses failures, tests corrective programs, and collects successful task and recovery rollouts for separate policies. During deployment, it monitors progress, invokes a learned or programmatic recovery, verifies scene restoration, and resumes execution. When no suitable recovery is available, a human demonstration resolves the failure and enters the learning loop, allowing the system to expand its recovery capabilities. Physical rollouts and human demonstrations are routed to the corresponding policy for DAgger training. Across six LIBERO-Pro settings and four MolmoSpaces categories, Recova achieves 78.8% and 64.9% mean success, compared with 71.7% and 38.0% for the strongest baselines. With parallel collection across four real-robot workstations, DAgger fine-tuning raises mean success from 23.8% to 77.5%, and recovery skills further raise it to 87.5%. Over four collection rounds on one task, observed human intervention falls from 87.5% to 0%. Together, these results show how agent-guided recovery turns failures into reusable capabilities, improving robustness while progressively reducing human intervention. Project page: https://www.liuisabella.com/Recova

Problem

• failure recovery
• autonomous robotic manipulation
• corrective behaviors
• task execution

Innovation

• Agent-Guided Recovery
• Digital Twin
• Failure Recovery
• Autonomous Robotic Manipulation
• DAgger Training