🤖 AI Summary
This work addresses the lack of provable correctness guarantees in visual pose estimation for safety-critical applications by proposing a certifiable pose estimation algorithm that integrates physics-driven geometric modeling with learning-based methods. The core innovation lies in introducing a Geometric Generative Model (GGM) combined with neural network reachability analysis to construct a multi-stage, certification-aware pipeline capable of delivering verifiable pose estimates and object detection under conditions ranging from unoccluded to complex occlusion scenarios. The approach is validated on both synthetic and real-world imagery—including event camera data—for planar objects such as traffic signs. Experimental results demonstrate that the estimated poses rigorously satisfy pre-specified certification error bounds, thereby achieving, for the first time, provably robust pose perception suitable for safety-critical deployment.
📝 Abstract
We consider the problem of vision-based pose estimation for autonomous systems. While deep neural networks have been successfully used for vision-based tasks, they inherently lack provable guarantees on the correctness of their output, which is crucial for safety-critical applications. We present a framework for designing certifiable neural networks (NNs) for perception-based pose estimation that integrates physics-driven modeling with learning-based estimation. The proposed framework begins by leveraging the known geometry of planar objects commonly found in the environment, such as traffic signs and runway markings, referred to as target objects. At its core, it introduces a geometric generative model (GGM), a neural-network-like model whose parameters are derived from the image formation process of a target object observed by a camera. Once designed, the GGM can be used to train NN-based pose estimators with certified guarantees in terms of their estimation errors. We first demonstrate this framework in uncluttered environments, where the target object is the only object present in the camera's field of view. We extend this using ideas from NN reachability analysis to design certified object NN that can detect the presence of the target object in cluttered environments. Subsequently, the framework consolidates the certified object detector with the certified pose estimator to design a multi-stage perception pipeline that generalizes the proposed approach to cluttered environments, while maintaining its certified guarantees. We evaluate the proposed framework using both synthetic and real images of various planar objects commonly encountered by autonomous vehicles. Using images captured by an event-based camera, we show that the trained encoder can effectively estimate the pose of a traffic sign in accordance with the certified bound provided by the framework.