Reliability Characterization for N-version Object Detection

2026-10-08

AI Summary

This study addresses the limitation of existing metrics in quantifying the diversity and consistency of N-version object detection, which constrains system reliability assessment. To overcome this, we propose Cov_OD (error coverage) and Cer_OD (prediction certainty) metrics that model independent detection outcomes without relying on voting strategies. By integrating multi-model and multi-frame fusion with customized statistical measures, the proposed approach is validated within an autonomous driving simulation environment. This work reveals the complementarity among version combinations, providing a theoretical foundation for selecting effective voting strategies. Furthermore, it successfully guides version screening, strategy optimization, and incremental system construction, thereby significantly enhancing perception reliability in autonomous driving applications.

Abstract

N-version object detection (OD) is an approach to diversifying detection results using multiple models or input frames and reducing detection errors by aggregating individual results. Diversity and consistency across multiple detection results are critical information for characterizing the reliability of possible configurations of N-version OD systems. However, existing performance metrics such as mAP and Accuracy fail to capture these factors, as they are defined solely on the final outcome after aggregation. To overcome this limitation, we propose two reliability metrics particularly defined for N-version OD, namely coverage of errors in OD (Cov_OD) and certainty of accurate prediction in OD (Cer_OD), which can be computed from individual detection results without relying on voting strategies. We empirically demonstrate the unique features of the proposed metrics through a case study of N-version OD for a vehicle in an autonomous-driving simulator. We show that the proposed metrics can be used for 1) selecting version combinations with complementary error characteristics, 2) choosing an effective voting strategy based on diversity and consistency profiles, and 3) guiding incremental construction of N-version OD systems via pairwise two-version analysis. These results highlight the importance of the metrics that can guide the design of reliable N-version OD applications.

Problem

• N-version object detection
• reliability characterization
• diversity
• consistency
• performance metrics

Innovation

• N-version Object Detection
• Reliability Metrics
• Diversity and Consistency
• Coverage of Errors (Cov_OD)
• Certainty of Accurate Prediction (Cer_OD)