Importance Sampling and PCA for Finding Failures in Commercial Autonomous Vehicles

2026-07-20

AI Summary

This work addresses the challenge of efficiently uncovering rare yet critical failure scenarios in commercial autonomous driving systems, which are often missed by conventional Monte Carlo simulation. The study proposes a novel approach that integrates adaptive stress testing (AST) with diffusion-based failure sampling (DiFS) to actively search for rare noise trajectories leading to collisions. Furthermore, principal component analysis (PCA) is employed to classify and diagnose distinct failure modes, establishing a closed-loop pipeline from failure discovery to perception defect localization. Evaluated on merging and cut-in scenarios, the method successfully identifies collision cases overlooked by traditional simulation. The extracted canonical noise trajectories consistently reproduce failures across identical or similar scenarios, demonstrating the approach’s effectiveness and transferability.

Abstract

Methods for discovering rare failures in autonomous systems have so far been demonstrated almost exclusively in simulations with simple, academic driving stacks, leaving open whether they generalize to the more robust planners used in commercial systems. We address this gap by applying two rare-event discovery algorithms to a commercial autonomous trucking stack. Adaptive stress testing (AST) uses reinforcement learning to search for the most likely noise trajectories leading to a simulated collision, while diffusion-based failure sampling (DiFS) trains a denoising diffusion model to sample a diverse set of failures. We show that both algorithms find simulated collisions during merge and cut-in maneuvers where traditional Monte Carlo simulation does not. To make these failures actionable, we introduce a statistical analysis based on principal component analysis (PCA) that classifies failures into common modes and identifies the timesteps that most influence the outcome. We cluster the principal components and invert the PCA transform to recover generalized noise trajectories, and show that these trajectories reproduce failures in identical and similar scenarios. This provides a path from failure discovery to systematic diagnosis of perception-level flaws.

Problem

• rare-event discovery
• autonomous vehicles
• failure detection
• commercial driving stacks
• simulation-based testing

Innovation

• Importance Sampling
• Principal Component Analysis
• Rare-Event Discovery
• Diffusion-Based Failure Sampling
• Autonomous Vehicle Validation