🤖 AI Summary
Autonomous driving systems (ADS) face a dual challenge in testing: naturally occurring traffic scenarios often lack sufficient risk, while synthetically generated critical scenarios suffer from low fidelity. This paper proposes the On-Demand Scenario Generation (OSG) framework, enabling controllable synthesis of diverse, naturalistic, and safety-critical traffic scenarios across a continuous risk spectrum. Our approach integrates real-world traffic data, multi-objective optimization, and heuristic search within a collaborative generation mechanism. Key innovations include: (i) the first quantifiable, tunable risk intensity controller; (ii) a synergistic generation pipeline unifying data-driven modeling, optimization, and search; and (iii) an end-to-end CARLA-based simulation infrastructure. Experiments demonstrate robust scenario generation across risk levels, uncover distinct ADS failure patterns under progressive risk exposure, and establish a new paradigm for objective, comparable, and systematic ADS safety and reliability evaluation.
📝 Abstract
The safety and reliability of Automated Driving Systems (ADS) are paramount, necessitating rigorous testing methodologies to uncover potential failures before deployment. Traditional testing approaches often prioritize either natural scenario sampling or safety-critical scenario generation, resulting in overly simplistic or unrealistic hazardous tests. In practice, the demand for natural scenarios (e.g., when evaluating the ADS's reliability in real-world conditions), critical scenarios (e.g., when evaluating safety in critical situations), or somewhere in between (e.g., when testing the ADS in regions with less civilized drivers) varies depending on the testing objectives. To address this issue, we propose the On-demand Scenario Generation (OSG) Framework, which generates diverse scenarios with varying risk levels. Achieving the goal of OSG is challenging due to the complexity of quantifying the criticalness and naturalness stemming from intricate vehicle-environment interactions, as well as the need to maintain scenario diversity across various risk levels. OSG learns from real-world traffic datasets and employs a Risk Intensity Regulator to quantitatively control the risk level. It also leverages an improved heuristic search method to ensure scenario diversity. We evaluate OSG on the Carla simulators using various ADSs. We verify OSG's ability to generate scenarios with different risk levels and demonstrate its necessity by comparing accident types across risk levels. With the help of OSG, we are now able to systematically and objectively compare the performance of different ADSs based on different risk levels.