From Stoplights to On-Ramps: A Comprehensive Set of Crash Rate Benchmarks for Freeway and Surface Street ADS Evaluation

2025-08-26

AI Summary

Existing autonomous driving system (ADS) safety evaluations lack a comprehensive crash-rate benchmark covering both freeways and urban roads, and fail to systematically quantify regional variations and crash severity impacts. Method: We propose the first U.S.-wide, multi-city ADS safety evaluation benchmark—stratified by road type (freeway/urban), geographic region, and crash severity—and develop a hierarchical statistical model integrating police-reported crash data with vehicle-miles-traveled (VMT) statistics. We introduce the freeway crash rate as a novel metric. Results: Atlanta’s freeway injury crash rate is 3.5× higher than Phoenix’s; performance in low-severity scenarios does not generalize to high-severity ones; location-specific evaluation criteria are essential. Our benchmark significantly improves statistical reliability of safety testing and provides quantitative guidance for determining verification mileage requirements under diverse safety objectives.

Technology Category

• Application Domains: Transportation
• Search and Optimization: Evaluation and Analysis
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

This paper presents crash rate benchmarks for evaluating US-based Automated Driving Systems (ADS) for multiple urban areas. The purpose of this study was to extend prior benchmarks focused only on surface streets to additionally capture freeway crash risk for future ADS safety performance assessments. Using publicly available police-reported crash and vehicle miles traveled (VMT) data, the methodology details the isolation of in-transport passenger vehicles, road type classification, and crash typology. Key findings revealed that freeway crash rates exhibit large geographic dependence variations with any-injury-reported crash rates being nearly 3.5 times higher in Atlanta (2.4 IPMM; the highest) when compared to Phoenix (0.7 IPMM; the lowest). The results show the critical need for location-specific benchmarks to avoid biased safety evaluations and provide insights into the vehicle miles traveled (VMT) required to achieve statistical significance for various safety impact levels. The distribution of crash types depended on the outcome severity level. Higher severity outcomes (e.g., fatal crashes) had a larger proportion of single-vehicle, vulnerable road users (VRU), and opposite-direction collisions compared to lower severity (police-reported) crashes. Given heterogeneity in crash types by severity, performance in low-severity scenarios may not be predictive of high-severity outcomes. These benchmarks are additionally used to quantify at the required mileage to show statistically significant deviations from human performance. This is the first paper to generate freeway-specific benchmarks for ADS evaluation and provides a foundational framework for future ADS benchmarking by evaluators and developers.

Problem

• Develops crash rate benchmarks for evaluating Automated Driving Systems
• Extends prior surface street benchmarks to include freeway crash risk
• Addresses geographic variations in crash rates to prevent biased safety evaluations

Innovation

• Freeway-specific crash rate benchmarks
• Location-specific statistical significance analysis
• Crash typology severity-based differentiation