safety metrics design

This paper addresses safety assurance for BMW’s first SAE Level 3 automated driving system, tackling risks arising from hardware/software faults, performance limitations, and specification gaps across the entire product lifecycle. Method: We propose the first holistic safety integrity framework integrating functional safety (ISO 26262) and safety of the intended functionality (ISO 21448). It uniquely couples Bayesian analysis, statistical learning, and the V-model, unifying design-of-experiments, real-world vehicle data, and expert knowledge. Scenario uncertainty is quantified via stochastic simulation and sensitivity analysis to rigorously assess residual risk. Contribution/Results: The framework enabled systematic, verifiable risk balancing and supported full development, series production, and global regulatory approval of the system. Safety arguments are transparent, independently verifiable, and underpinned by quantitative assurance—establishing a new benchmark for certification-ready autonomy safety engineering.

To address the lack of clear mapping between safety objectives and performance metrics for machine learning components in aviation—such as object detection models in emergency braking systems—this paper proposes a system safety–driven method for deriving verifiable performance requirements. First, safety-critical scenarios are identified via Functional Hazard Analysis (FHA) and Fault Tree Analysis (FTA). Second, a behavioral abstraction model of the ML component is constructed to quantitatively decompose top-level safety requirements (e.g., maximum allowable failure probability) into concrete neural network performance constraints (e.g., minimum detection accuracy, robustness thresholds against adversarial perturbations). Finally, applicable assumptions are explicitly defined, and the impact of verification deviations is quantified. The method establishes a traceable, rigorously verifiable linkage from safety goals to model-level metrics, providing both theoretical foundations and a practical framework for airworthiness certification of ML components in safety-critical domains.

This study addresses significant shortcomings in existing AI safety benchmarks, which inadequately assess advanced AI systems across technical, cognitive, and sociotechnical dimensions. Through a systematic review of 210 benchmarks, the work introduces classical risk management principles and measurement theory into the design of AI safety evaluations for the first time. It proposes a framework centered on measurability boundary analysis and probabilistic metric design. Combining systematic literature review, theoretical modeling, and hybrid evaluation methodologies, the research develops an actionable roadmap and a practical checklist for benchmark development. Empirical validation demonstrates the efficacy of the proposed approach, offering researchers and practitioners a rigorous, operational guide to constructing more robust and meaningful AI safety benchmarks.

Current cyber-physical systems (CPS) in vehicular environments lack quantitative, experimentally grounded methods for assessing network resilience. Method: This study constructs an experimental testbed replicating real-world truck operational conditions and conducts multiple rounds of malware injection attacks, simultaneously collecting network- and physical-layer data on resistance and recovery behaviors. Contribution/Results: We introduce the novel concept of “bonware” to holistically characterize both cybersecurity defense capability and physical resilience, formalized via an analytically tractable mathematical model. We further define and extract experimentally identifiable, quantitative resilience metrics—termed elastic features—for the first time. Sensitivity analysis confirms these metrics exhibit significant discriminability with respect to attack intensity, defensive strategies, and physical redundancy. This work bridges a critical gap by advancing vehicular CPS resilience from qualitative description to quantifiable, comparable, and optimizable measurement.

Evaluating coverage adequacy of scenario libraries for Automated Driving System (ADS) type-approval remains challenging, particularly in ensuring both comprehensive Operational Design Domain (ODD) coverage and faithful representation of intrinsic diversity in real-world driving data. Method: This paper proposes a computable, two-layer coverage metric: (i) quantifying ODD coverage sufficiency, and (ii) assessing representational completeness of underlying driving data diversity. Our approach integrates statistical analysis of the HighD dataset, scenario clustering with ODD-dimensional mapping, coverage modeling, and gap identification algorithms. Contribution/Results: Evaluated on over 200,000 real-world traffic scenarios across 10 categories, the method achieves up to 100% ODD coverage under specific conditions and precisely identifies missing scenario types and data patterns. To our knowledge, this is the first work to jointly and quantitatively assess ODD coverage and data diversity coverage, establishing a reproducible, verifiable evaluation paradigm for scenario library construction.

This work addresses the limitations of traditional Failure Modes, Effects, and Diagnostic Analysis (FMEDA) in automotive ASIC functional safety verification, where expert judgment is used to estimate failure mode distributions and diagnostic coverage without quantifying associated uncertainties, thereby compromising reliability. For the first time, error propagation theory is systematically integrated into FMEDA to construct uncertainty models for both failure mode distributions and diagnostic coverage. This enables quantitative computation of the maximum deviations and confidence intervals for the Single-Point Fault Metric (SPFM) and Latent Fault Metric (LFM). Furthermore, an Error Importance Indicator (EII) is introduced to trace the key contributors driving overall uncertainty. The proposed approach significantly enhances the transparency and credibility of FMEDA, offering a scientifically rigorous and quantifiable foundation for compliance with ISO 26262.

This study addresses the absence of standardized safety capability thresholds among leading AI developers, which hinders third-party verification and comparison and risks a race to the bottom in safety standards. The work proposes the first coordinated threshold framework spanning three risk domains: cyber misuse, biological misuse, and autonomous AI development. For misuse risks, the framework centers on expected harm, integrating risk pathway analysis with release condition modeling; for autonomous development risks, it dynamically assesses thresholds based on the pace of AI progress. By differentiating evaluation logics across risk domains, the framework addresses gaps in existing empirical research and establishes a unified, comparable, and verifiable system of AI safety thresholds, offering a methodological foundation for regulatory and industry standards.

This study addresses the limitations of current safety evaluations, which predominantly rely on isolated multiple-choice setups and overlook the real-world impact of agent scaffolding on model safety. Through a large-scale controlled experiment (N = 62,808), we systematically assess four scaffolding architectures—including Map-Reduce—across evaluation formats (open-ended vs. multiple-choice) on state-of-the-art language models. We find that evaluation format exerts a far stronger influence on safety scores than scaffolding effects themselves. Critically, strong model–scaffolding interactions lead to complete reversals in safety rankings across benchmarks (G = 0.000). Employing preregistration, evaluator blinding, TOST equivalence testing, and generalizability analyses, we demonstrate that safety must be evaluated for each specific model–deployment configuration: Map-Reduce significantly reduces safety (NNH = 14), whereas other architectures remain equivalent within ±2 percentage points.

This work addresses the lack of systematicity and defensibility in current AI safety assurance approaches, which often inadequately adapt principles from traditional safety engineering. To remedy this, the paper proposes a novel framework integrating structured assurance methodologies from safety-critical domains such as aerospace and nuclear energy. The framework cohesively combines risk assessment, formal modeling, and structured argumentation to critically reassess and enhance prevailing paradigms for constructing safety cases in the AI alignment community. Through case studies on deceptive alignment and CBRN (chemical, biological, radiological, and nuclear) capabilities, the authors demonstrate how this approach yields a more holistic, rigorous, and practical foundation for AI safety assurance. The resulting methodology offers an actionable and defensible pathway for the safe deployment of high-stakes AI systems.