BayesContact: Uncertain Pose Estimation via Visuo-Tactile Proposals and Simulation-based Inference

2026-07-17

AI Summary

This work addresses the challenge of accurate object pose estimation in contact-rich manipulation tasks, where purely vision-based methods often fall short and existing vision-tactile fusion approaches suffer from reliance on costly offline training and poor generalization. The paper proposes BayesContact, a novel framework that introduces simulation-based inference into vision-tactile pose estimation for the first time. It employs particle filtering to maintain an online belief over object pose, fusing depth images with contact evidence derived from force/torque measurements. Crucially, a physics simulation-driven forward model computes observation likelihoods, enabling active probing and pose refinement without retraining. Experiments demonstrate that BayesContact significantly improves pose observability and task success rates—by approximately 30%—over vision-only baselines in both simulated and real-world robotic peg-in-hole tasks.

Abstract

Contact-rich manipulation requires pose estimates that are often more accurate than what depth-only sensing provides. Existing methods, relying on vision and contact, employ costly offline training procedures that need to be retrained for new environments and geometries. We propose BayesContact, a Simulation-Based Inference framework for visuo-tactile pose estimation in peg-in-hole insertion. BayesContact maintains a particle belief over object pose and fuses depth observations with force/torque-derived contact evidence. We employ simulation based forward models to approximate these observation likelihoods. For each pose hypothesis, a renderer predicts depth measurements and a physics simulator predicts contact outcomes under guarded probing actions; both are scored against real observations to update the belief. The resulting multimodal belief also enables information-gain-based probing for active disambiguation. Across simulated geometries and real-robot experiments, BayesContact improves pose observability and insertion success over vision-only inference by 30%

Problem

• pose estimation
• visuo-tactile sensing
• contact-rich manipulation
• peg-in-hole insertion
• simulation-based inference

Innovation

• Simulation-Based Inference
• Visuo-Tactile Fusion
• Pose Estimation
• Active Probing
• Particle Belief