SV2V-RSim: A Comprehensive Benchmark for Self-Selective V2V Cooperative Perception with Near-Realistic Data

2026-09-26

AI Summary

This study addresses the limitations of existing vehicle-to-vehicle (V2V) cooperative perception datasets, including restricted agent scale, static vehicle selection strategies, and significant simulation-to-real domain gaps. To this end, we propose a large-scale, high-fidelity multimodal simulation dataset built upon Unreal Engine 5, alongside a novel adaptive vehicle selection module termed SVA. By integrating LiDAR and RGB sensor data with 3D bounding box annotations, the proposed dataset effectively mitigates the sim-to-real domain shift. Furthermore, the SVA module dynamically optimizes transmitting agents, achieving an adaptive trade-off between detection performance and communication bandwidth. Experimental results demonstrate that our approach yields superior performance–bandwidth trade-offs in 3D object detection tasks, validating both the high fidelity and practical utility of the proposed dataset and selection strategy.

Abstract

Vehicle-to-Vehicle (V2V) cooperative perception enhances autonomous driving by enabling vehicles to share information beyond their direct line of sight. However, existing V2V datasets are limited by a small number of participating agents, static collaborator selection strategies, and a significant domain gap between simulated and real-world environments. To overcome these challenges, we introduce SV2V-RSim, a large-scale, multi-modal, near-realistic simulation dataset engineered to elevate agent diversity and realism. Additionally, we present the Select Vehicles Adaptively (SVA) module, which optimizes collaborator selection to balance perception performance against communication bandwidth constraints. Our dataset is generated using the Unreal Engine 5-based simulator that integrates high-fidelity 3D assets, diverse environments, and intricate traffic scenarios. All vehicles within a specified range of the ego vehicle are equipped with sensor suites, enabling dynamic and adaptive collaborator selection. SV2V-RSim encompasses four maps, four weather conditions, six time periods from sunrise to night, 203K LiDAR frames, 402K RGB frames, and 788K annotated 3D bounding boxes across 17 object classes, supporting a range of cooperative perception tasks such as 3D object detection, segmentation, and depth estimation. Benchmarking on recent cooperative perception algorithms demonstrates that SVA achieves a superior performance-bandwidth trade-off, while sim-to-real experiments and No-Reference Image Quality Assessment validate the dataset's high realism and practical effectiveness. Our dataset and code will be publicly available.

Problem

• V2V cooperative perception
• collaborator selection
• domain gap
• benchmark dataset

Innovation

• Cooperative Perception
• Vehicle-to-Vehicle (V2V)
• Near-Realistic Simulation
• Adaptive Collaborator Selection
• SVA Module