SimBEV2X: A Large-Scale Dataset and Data Generation Tool for Multi-Task Vehicle-to-Everything Cooperative Perception

2026-07-26

AI Summary

This work addresses the scarcity of large-scale, multimodal, and multitask datasets that hinders the development of unified spatial representation methods—such as bird’s-eye-view (BEV)—for vehicle-to-everything (V2X) perception. To this end, we propose SimBEV2X, a synthetic data generation framework built on CARLA that automatically constructs diverse driving scenarios with multiple agents and heterogeneous sensors, providing ground-truth annotations for 3D object detection, BEV segmentation, and semantic voxelization. We release the largest V2X perception dataset to date, comprising 258 scenarios, 102,200 frames, over 3 million images, and 27 million 3D bounding boxes. Furthermore, we introduce CoBEVFusion, a novel architecture featuring Fusion Axial Attention (FAX) to enable context-aware feature aggregation across agents, which significantly outperforms existing methods on this benchmark.

Abstract

Cooperative perception through vehicle-to-everything (V2X) communication can overcome the inherent physical limitations of individual autonomous vehicles, such as occlusions and limited sensor range. However, the development of robust V2X algorithms, particularly those relying on unified spatial representations like bird's-eye view (BEV) representation, is hampered by the lack of large-scale, multi-modal, multi-task datasets. Moreover, collecting and annotating a large set of synchronized, real-world multi-agent data is prohibitively expensive. This has resulted in a landscape where existing V2X datasets are notably limited in both size and scope. To overcome this, we introduce SimBEV2X, an advanced synthetic data generation tool built on the CARLA simulator. SimBEV2X automatically creates randomized driving scenarios to collect multi-modal sensor data alongside various types of ground truth including 3D bounding boxes with unique track IDs, HD map information, BEV segmentation maps, and semantic occupancy voxel grids from both vehicles and RSUs. We also present the SimBEV2X dataset, the largest V2X perception dataset to date. The dataset comprises 258 scenes, each involving up to 8 connected vehicles and up to 4 RSUs across a variety of road networks. The SimBEV2X dataset is an order of magnitude larger than existing V2X datasets and contains 102,200 frames, 588,520 lidar point clouds, more than 3 million images, over 27 million bounding boxes, and a comprehensive set of other annotations. Finally, we establish a strong baseline on the SimBEV2X dataset using CoopDet3D and propose CoBEVFusion, a novel architecture that combines CoopDet3D with fused axial attention (FAX) for context-aware multi-agent feature aggregation, resulting in superior performance. SimBEV2X, the SimBEV2X dataset, and CoBEVFusion are available at https://simbev2x.org and https://github.com/GoodarzMehr/SimBEV2X.

Problem

• V2X
• cooperative perception
• BEV representation
• large-scale dataset
• multi-task

Innovation

• synthetic data generation
• bird's-eye view (BEV) representation
• multi-agent cooperative perception
• fused axial attention
• V2X dataset