X4Val: Learning Neural Surrogates for Variance-Reduced Policy Evaluation

2026-06-03

AI Summary

Accurately and efficiently evaluating the performance of learning-based robotic systems remains highly challenging in the absence of large-scale real-world data. This work proposes a general evaluation framework that, for the first time, enables variance-reduced performance estimation using heterogeneous, multi-source data—such as simulation outputs and historical logs—without requiring paired samples. By constructing a shared representation space and a transferable predictor, the method integrates neural surrogate models with control variates to substantially improve sample efficiency. Empirical results on autonomous driving and robotic manipulation tasks demonstrate up to a 38.4% reduction in estimator variance, consistently outperforming strong existing baselines.

Abstract

Rigorous evaluation of learning-based robotic systems is an essential prerequisite for deployment. However, real-world test data is expensive to gather; moreover, in a typical iterative development context, data gathered from the latest policy is necessarily limited in scale. This motivates evaluation methodologies that make use of heterogeneous data sources, including simulation, historical policy logs, and data collected from related platforms or environments. While such auxiliary data are abundant and inexpensive, they are generally not directly representative of real-world outcomes -- for example, performance in simulation may differ substantially from performance in the real world -- making their principled use for high-confidence performance estimation challenging. In this paper, we introduce X4Val, a general framework for variance-reduced real-world metric estimation in the presence of non-paired, multi-domain data. X4Val embeds samples from real and auxiliary domains into a shared representation space and learns a transferable predictor of real-world metrics; this learned predictor is then incorporated into a control-variates estimator, enabling variance reduction even when paired samples are unavailable. We provide theoretical analysis and empirical evaluations on autonomous driving and real-world robot manipulation tasks, domains across which X4Val achieves up to 38.4% variance reduction and demonstrates consistent improvements over strong baselines. These results show that non-paired, heterogeneous data can be leveraged to substantially improve the sample efficiency of rigorous robotic system validation.

Problem

• policy evaluation
• variance reduction
• heterogeneous data
• robotic system validation
• multi-domain data

Innovation

• variance reduction
• multi-domain data
• control variates
• neural surrogates
• policy evaluation