🤖 AI Summary
Evaluating coverage adequacy of scenario libraries for Automated Driving System (ADS) type-approval remains challenging, particularly in ensuring both comprehensive Operational Design Domain (ODD) coverage and faithful representation of intrinsic diversity in real-world driving data.
Method: This paper proposes a computable, two-layer coverage metric: (i) quantifying ODD coverage sufficiency, and (ii) assessing representational completeness of underlying driving data diversity. Our approach integrates statistical analysis of the HighD dataset, scenario clustering with ODD-dimensional mapping, coverage modeling, and gap identification algorithms.
Contribution/Results: Evaluated on over 200,000 real-world traffic scenarios across 10 categories, the method achieves up to 100% ODD coverage under specific conditions and precisely identifies missing scenario types and data patterns. To our knowledge, this is the first work to jointly and quantitatively assess ODD coverage and data diversity coverage, establishing a reproducible, verifiable evaluation paradigm for scenario library construction.
📝 Abstract
Automated Driving Systems (ADSs) have the potential to make mobility services available and safe for all. A multi-pillar Safety Assessment Framework (SAF) has been proposed for the type-approval process of ADSs. The SAF requires that the test scenarios for the ADS adequately covers the Operational Design Domain (ODD) of the ADS. A common method for generating test scenarios involves basing them on scenarios identified and characterized from driving data.This work addresses two questions when collecting scenarios from driving data. First, do the collected scenarios cover all relevant aspects of the ADS’ ODD? Second, do the collected scenarios cover all relevant aspects that are in the driving data, such that no potentially important situations are missed? This work proposes coverage metrics that provide a quantitative answer to these questions.The proposed coverage metrics are illustrated by means of an experiment in which over 200 000 scenarios from 10 different scenario categories are collected from the HighD data set. The experiment demonstrates that a coverage of 100% can be achieved under certain conditions, and it also identifies which data and scenarios could be added to enhance the coverage outcomes in case a 100% coverage has not been achieved. Whereas this work presents metrics for the quantification of the coverage of driving data and the identified scenarios, this paper concludes with future research directions, including the quantification of the completeness of driving data and the identified scenarios.