Latent Two-Sample Testing for Fair Autonomous Vehicle Road Evaluation

2026-08-02

AI Summary

Autonomous driving evaluation is prone to bias due to variations in testing environments and interference from high-dimensional, unstructured data. To address this, this work proposes a distribution-aware evaluation framework that first maps on-road test data into a compact latent space using a structured latent variable model. It then introduces a two-stage two-sample testing procedure to accurately detect and localize distributional shifts, followed by calibration via importance sampling. This approach uniquely integrates latent space modeling with two-sample hypothesis testing, substantially reducing both evaluation bias and estimation error of performance metrics. Empirical validation on both synthetic and real-world datasets demonstrates the effectiveness and robustness of the proposed method.

Abstract

With the rapid advancement of autonomous vehicle (AV) systems, fast and reliable iteration through road testing has become increasingly critical. However, changes in testing environments make it difficult to disentangle true performance differences between AV versions from extraneous environmental variations, undermining fair and reliable evaluation. This challenge is further compounded by the high-dimensional and unstructured nature of large-scale road testing data, for which effective analysis and comparison methods remain limited. In this work, we address these challenges by introducing a principled framework for distribution-aware AV evaluation. We first learn structured latent representations that map high-dimensional, unstructured road testing data into a compact latent space, enabling effective characterization of scenario distributions. Building on this representation, we propose a two-stage two-sample testing framework that (i) detects and localizes distributional shifts between testing datasets and (ii) calibrates these shifts via importance sampling to reduce evaluation bias and metric estimation error. Experiments on synthetic and real-world road testing data demonstrate the effectiveness of the proposed method.

Problem

• autonomous vehicle evaluation
• distribution shift
• two-sample testing
• fairness
• road testing data

Innovation

• latent representation
• two-sample testing
• distribution shift
• importance sampling
• autonomous vehicle evaluation