Towards more efficient quantitative safety validation of residual risk for assisted and automated driving

2025-06-12

AI Summary

Under ISO 21448 (SOTIF), quantitative validation of residual risks for ADAS/ADS remains inefficient, while real-world Field Operational Tests (FOTs) incur prohibitively high costs. Method: This study systematically evaluates existing FOT reduction techniques and—firstly—establishes a standardized, reusable argumentation component model aligned with ISO 214448, alongside a benchmark Quantitative Safety Verification and Risk Reduction (QSVRR) model for AEB. It further proposes a four-dimensional evaluation framework—assessing quantifiability, validity threats, critical missing elements, and black-box compatibility—to rigorously analyze current reduction methods. Results: The analysis reveals that all existing FOT reduction approaches suffer from irreparable deficiencies in essential safety assurance aspects, thereby confirming FOT’s irreplaceable role in macro-level safety validation. The work delivers a standardized, scalable safety verification paradigm for higher-level automated driving and explicitly identifies key research gaps.

Technology Category

• Application Domains: Mobility, Driving & Flight
• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

The safety validation of Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS) increasingly demands efficient and reliable methods to quantify residual risk while adhering to international standards such as ISO 21448. Traditionally, Field Operational Testing (FOT) has been pivotal for macroscopic safety validation of automotive driving functions up to SAE automation level 2. However, state-of-the-art derivations for empirical safety demonstrations using FOT often result in impractical testing efforts, particularly at higher automation levels. Even at lower automation levels, this limitation - coupled with the substantial costs associated with FOT - motivates the exploration of approaches to enhance the efficiency of FOT-based macroscopic safety validation. Therefore, this publication systematically identifies and evaluates state-of-the-art Reduction Approaches (RAs) for FOT, including novel methods reported in the literature. Based on an analysis of ISO 21448, two models are derived: a generic model capturing the argumentation components of the standard, and a base model, exemplarily applied to Automatic Emergency Braking (AEB) systems, establishing a baseline for the real-world driving requirement for a Quantitative Safety Validation of Residual Risk (QSVRR). Subsequently, the RAs are assessed using four criteria: quantifiability, threats to validity, missing links, and black box compatibility, highlighting potential benefits, inherent limitations, and identifying key areas for further research. Our evaluation reveals that, while several approaches offer potential, none are free from missing links or other substantial shortcomings. Moreover, no identified alternative can fully replace FOT, reflecting its crucial role in the safety validation of ADAS and ADS.

Problem

• Efficient safety validation for ADAS and ADS
• Reducing impractical testing efforts in FOT
• Quantifying residual risk per ISO 21448 standards

Innovation

• Systematic evaluation of FOT Reduction Approaches
• Derived models based on ISO 21448 standards
• Assessed approaches using four key criteria