CATPlan: Loss-based Collision Prediction in End-to-End Autonomous Driving

2025-03-10

AI Summary

To address the lack of trajectory uncertainty modeling and real-time collision risk assessment in end-to-end autonomous driving, this paper proposes CATPlan—a lightweight, plug-and-play module for risk quantification. CATPlan is the first to introduce loss prediction into planning uncertainty modeling: it decodes motion and planning embeddings to directly predict per-trajectory collision loss, requiring no additional annotations and no modification to the backbone network. Furthermore, we integrate Neural Radiance Fields (NeRF) to construct NeuroNCAP, a neural closed-loop evaluation benchmark. Experiments demonstrate that CATPlan achieves a 54.8% improvement in mean collision detection accuracy over the Gaussian Mixture Model (GMM) baseline on NeuroNCAP, significantly enhancing system safety and robustness.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In recent years, there has been increased interest in the design, training, and evaluation of end-to-end autonomous driving (AD) systems. One often overlooked aspect is the uncertainty of planned trajectories predicted by these systems, despite awareness of their own uncertainty being key to achieve safety and robustness. We propose to estimate this uncertainty by adapting loss prediction from the uncertainty quantification literature. To this end, we introduce a novel light-weight module, dubbed CATPlan, that is trained to decode motion and planning embeddings into estimates of the collision loss used to partially supervise end-to-end AD systems. During inference, these estimates are interpreted as collision risk. We evaluate CATPlan on the safety-critical, nerf-based, closed-loop benchmark NeuroNCAP and find that it manages to detect collisions with a $54.8%$ relative improvement to average precision over a GMM-based baseline in which the predicted trajectory is compared to the forecasted trajectories of other road users. Our findings indicate that the addition of CATPlan can lead to safer end-to-end AD systems and hope that our work will spark increased interest in uncertainty quantification for such systems.

Problem

• Estimating uncertainty in autonomous driving trajectory predictions.
• Improving collision detection in end-to-end autonomous systems.
• Enhancing safety through uncertainty quantification in AD systems.

Innovation

• CATPlan module predicts collision loss
• Uses motion and planning embeddings
• Improves collision detection precision