Audit Before You Commit: Locating Belief Failures in Active Identification for One-Shot Manipulation

2026-09-24

AI Summary

This study addresses decision-making failures in single-shot irreversible robotic operations caused by insufficient belief coverage and biased failure models. To this end, we propose a dual conditional independence auditing framework that decouples belief truth-value coverage from failure model accuracy. By integrating particle filtering with conformal prediction to calibrate localization observation errors and optimize probing strategies, the method exposes the limitations of conventional confidence gating for precisely identifying execution or observation defects. Experimental results demonstrate that correcting boundary errors reduces the failure rate from 0.354 to 0.112. Furthermore, the proposed approach exhibits cross-engine generalization capabilities and is successfully validated on a physical robotic manipulator.

Abstract

A robot that probes a few times before one irreversible action, such as tapping a surface before inserting a peg, must decide when the evidence is enough to commit. We argue that this decision rests on two conditions that existing methods do not separate: the belief must still cover the truth in the coordinate that decides the action, and the failure model that scores actions must track realized failure. We audit both conditions separately, offline and with ground truth, on a deployed probe-then-commit pipeline: a particle belief, a scenario failure score, and one commit. On simulated insertion, more taps sharpen the belief while the truth leaves its support on 16.9% of episodes and the failure score turns optimistic by 0.31. Conformal calibration restores coverage but not the decision: confidently wrong instances still pass a confidence gate. The audit's signatures instead point at the observation model, where a hand scan finds a 2.1 mm error in the tap boundary. Correcting that one number cuts failure from 0.354 to 0.112 on untouched instances and transfers unrefitted to a second engine, while in a third engine the same audit suggests an execution-model mismatch instead. Across seven task families in three engines, a few probes at a fixed executor reduce miss or failure. On a physical arm inserting a tool into a rigid pocket by touch, the gain and the audit's two conditions reproduce, and replaying the recorded taps under an injected model error shows the audit's signature on real data. Additional materials are available at https://sites.google.com/view/auditbeforeyoucommit.

Problem

• one-shot manipulation
• belief failure
• active identification
• probe-then-commit
• failure localization

Innovation

• Active Identification
• One-Shot Manipulation
• Belief Audit
• Conformal Calibration
• Observation Model