Detecting Pose Estimation Failures via Keypoint Self-Consistency

2026-08-04

AI Summary

This work addresses the unreliability of existing keypoint-based pose estimation methods, which often neglect geometric constraints inherent to object shape, leading to undetectable failures. To overcome this limitation, the authors propose a failure detection approach that does not rely on keypoint uncertainty estimates. Instead, it leverages handcrafted geometric consistency features—specifically pairwise keypoint distances, reprojection errors, and consistency between rendered and observed masks—to characterize spatial relationships among 2D keypoints. These features are fed into a logistic regression classifier to determine whether a given pose estimate has failed. Experimental results demonstrate that the proposed method significantly outperforms existing confidence-based failure detection schemes, such as conformal keypoint prediction, offering superior reliability and practical utility in real-world applications.

Abstract

One common approach to pose estimation involves predicting object keypoints in an image, followed by using Perspective-n-Point algorithms to compute the object's rotation and translation relative to the camera. While rotations preserve object shapes, this property is often neglected in keypoint-based pose estimation methods, where keypoints are typically predicted independently from each other. As imprecise keypoint predictions negatively affects pose estimation accuracy, it also limits its reliability in downstream tasks. In this work, we explore whether such inaccurate pose estimates can be identified by simply examining spatial locations between 2D keypoints. We propose a set of hand-crafted geometric features that capture the self-consistency of keypoint predictions, including pairwise distances, reprojection consistency, as well as render and mask consistency. Despite its simplicity, a logistic regression classifier trained on these features reliably detects pose estimation failures, outperforming confidence-based approaches like conformal keypoint predictions that rely solely on keypoint uncertainty.

Problem

• pose estimation
• keypoint prediction
• geometric consistency
• failure detection
• Perspective-n-Point

Innovation

• keypoint self-consistency
• pose estimation failure detection
• geometric consistency
• Perspective-n-Point
• confidence calibration