SceneFactory-3D: Lifting 2D Traffic Scenes into 3D Physical Counterfactuals for Scalable Physically Grounded Safety Evaluation

2026-10-02

AI Summary

This study addresses the limitation of existing driving simulators that neglect tire–road physical interactions, hindering the evaluation of adverse road conditions on vehicle control and traffic flow. We propose a physics-driven, multi-agent driving simulator leveraging GPU-batched computation. By incorporating spatially varying friction, 3D height fields, and rigid contact models, the framework enables parallel counterfactual evaluation under fixed controllers while exclusively altering road surface conditions. Computational efficiency is achieved through suspension- and friction-limited force models coupled with per-wheel contact simulation. Experiments demonstrate that reduced friction coefficients decrease the safe passage rates of learned policies by 6 to 90 percentage points while significantly increasing near-collision frequencies. These results validate the effectiveness of the proposed framework for scalable safety assessment in autonomous driving.

Abstract

Scalable driving simulators typically execute vehicle commands using prescribed behavioral or kinematic rules, overlooking the physics of tire-road interfaces, thereby limiting their ability to capture how adverse road and environmental conditions alter vehicle execution and propagate through traffic. To address this limitation, we present SceneFactory-3D, a GPU-batched, physics-grounded multi-agent driving simulator. Vehicles execute acceleration and steering commands via suspension- and friction-limited forces evaluated at each wheel-contact point. Spatially varying friction, per-world 3D heightfields, gravity, and rigid contact consistently govern wheel motion and chassis collisions. Per-world terrain isolation and GPU batching enable SceneFactory-3D to run matched physical counterfactuals in parallel: traffic scenario setup and vehicle controllers remain fixed while only the road condition changes, enabling the resulting closed-loop effects to be evaluated across parallel worlds. To demonstrate the advantage of the SceneFactory-3D-enabled counterfactual evaluation, we conduct an empirical study on vehicle controllers' sensitivity to road conditions. We study three learned-policy families on 1,024 matched 12-vehicle worlds per condition, and two classical planners on a shared 32-world subset, across 21 friction and grade conditions. When friction drops from 1.0 to 0.18, the share of vehicles that clear the work zone safely falls by 6 to 90 percentage points across learned policies (18-19 for classical planners), and near-collision situations become more frequent for every learned policy. Code: https://github.com/SmallWorldLab/SceneFactory_3D

Problem

• driving simulator
• physics-grounded simulation
• safety evaluation
• tire-road physics
• counterfactual evaluation

Innovation

• Physics-grounded simulation
• GPU-batched parallelism
• Physical counterfactuals
• Tire-road friction modeling
• Multi-agent driving simulator