Plan-Conditioned Imitation for Robust Object Retrieval under Self-Occlusion in Dense Clutter

2026-09-29

AI Summary

This study addresses the challenges of object retrieval failures and frequent replanning caused by robotic arm self-occlusion in densely cluttered environments. To this end, we propose TRACE, a framework that generates fixed nominal trajectories via digital twins and employs an open-loop execution strategy. The method optimizes student state supervision through plan-conditioned imitation learning combined with the DAgger algorithm, eliminating the need for online teacher queries or additional simulation rollouts. Experimental evaluations demonstrate that TRACE achieves success rates of 90.7% and 90.0% in simulated and real-world scenarios, respectively. Furthermore, it reduces the average execution time to 67.3 seconds while significantly decreasing the number of perception retries, thereby enabling efficient and robust object retrieval in complex settings.

Abstract

Retrieving objects from dense clutter requires rearrangement during which the manipulator can occlude objects while moving them. Repeated arm withdrawals to restore visibility interrupt execution. We introduce TRACE, a plan-conditioned imitation framework for retrieval under self-occlusion. A single unoccluded observation initializes a digital twin, where a privileged teacher generates a fixed nominal rollout. A recurrent student combines local rollout context, partial object observations, and proprioception to select actions that can correct deviations from the prediction. Behavior cloning initializes the student; DAgger refines it with teacher labels on student-visited states. The rollout remains fixed throughout execution, so the deployed student needs neither online teacher queries nor additional simulator rollouts during pushing. On 511 simulation test scenes, TRACE achieves 90.7% success versus 43.4% for nominal replay and 96.7% for the privileged closed-loop teacher. At a matched 26,373-label budget, student-state supervision achieves 87.8% versus 66.7% for expert-only cloning, demonstrating gains beyond additional labels. On a UR5e, TRACE achieves 90.0% success versus 95.0% for the closed-loop teacher, while reducing total execution time from 192.7 s to 67.3 s. It avoids the teacher's 16.8 sensing-related arm retractions per trial during pushing, retaining a final withdrawal for graspability evaluation. Code and data will be released at: https://trace-retrieval.github.io.

Problem

• object retrieval
• dense clutter
• self-occlusion
• robotic manipulation
• rearrangement

Innovation

• Plan-Conditioned Imitation
• Digital Twin
• Self-Occlusion
• DAgger
• Dense Clutter