Problem
Research questions and friction points this paper is trying to address.
Robot Manipulation
Failure Analysis
Proprioception
Failure Onset Localization
Vision-Language Models
ProTracer: Proprioception-Guided Failure Diagnosis in Robot Manipulation
🤖 AI Summary
本文提出ProTracer框架，利用视觉-语言模型与本体感受信号无训练地分析机器人操作失败原因及定位失败起始时刻。
📝 Abstract
This paper presents a comprehensive framework for robot manipulation failure analysis that includes binary failure detection, failure categorization, explanation generation, and the additional capability of failure onset localization, which aims to identify the earliest moment at which a robot execution deviates from a valid task-completion trajectory and is ultimately followed by task failure. To address these tasks, we propose ProTracer, a training-free framework that leverages existing Vision-Language Models (VLMs) together with proprioceptive signals for failure analysis. Our method uses proprioceptive dynamics to identify temporally informative action boundaries and converts richer robot-state signals into structured natural-language descriptions that can be jointly analyzed together with visual observations by the VLM. This design combines the temporal precision of proprioceptive signals with the multimodal reasoning capabilities of modern VLMs without requiring additional model training. We further introduce FailTime, a benchmark with synchronized visual and proprioceptive observations for evaluating conventional failure diagnosis tasks as well as failure onset localization. Experiments demonstrate that ProTracer achieves strong performance across both conventional failure diagnosis tasks and the newly introduced failure onset localization task, highlighting the importance of proprioceptive reasoning for fine-grained temporal failure analysis.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Proprioception-Guided
Failure Onset Localization
Vision-Language Models
Temporal Precision
🔎 Similar Papers
No similar papers found.
