Problem
Research questions and friction points this paper is trying to address.
end-to-end autonomous driving
safety evaluation
pedestrian risky motion
photorealistic scenario generation
controllability
Controllable and Photorealistic Pedestrian Risky Motion Generation for End-to-End Driving Safety Evaluation
🤖 AI Summary
This study addresses the scarcity of controllable and photorealistic sensor-level visual data for end-to-end autonomous driving in rare, hazardous pedestrian-vehicle interaction scenarios. To this end, it proposes a novel rendering framework that integrates trajectory-level conflict synthesis with 3D Gaussian Splatting (3DGS), coupled with a text-conditioned action diffusion model, enabling the controllable generation of photorealistic, multi-view dynamic hazardous pedestrian scenes directly from textual descriptions. Furthermore, this work constructs the HazardPed dataset alongside a comprehensive evaluation benchmark. Extensive experiments reveal a substantial performance degradation in mainstream end-to-end driving models when confronted with hazardous behaviors, with average scores declining precipitously from 88.8 to 47.4. All associated code and datasets have been made publicly available to facilitate future research.
📝 Abstract
Evaluating end-to-end autonomous driving under rare, safety-critical vehicle-pedestrian interactions requires photorealistic, sensor-level scenarios. However, trajectory-based scenario generators cannot synthesize raw visual observations, whereas video-based approaches lack controllability. To bridge this gap, we present ControlPed, a novel framework that combines trajectory-level conflict synthesis with 3D Gaussian Splatting (3DGS) to generate photorealistic, motion-controllable safety-critical scenarios. Built upon HazardPed, a dataset derived from 10,352 traffic videos comprising 422 conflict trajectories, HD maps, and 857 annotated 3D human motions, ControlPed first generates conflict trajectories, lifts them into 3D human motion sequences via text-conditioned motion diffusion, and finally renders multi-view sensor observations using animatable 3DGS avatars. Safety evaluation in 88 rendered photorealistic scenarios reveals that seven leading end-to-end driving models suffer a severe performance drop, with their mean HDScore plunging from 88.8 to 47.4, exposing major failure modes under dangerous pedestrian behaviors. The dataset and testing benchmarks will be released to facilitate safety assessment of vehicle-pedestrian interactions.
Innovation
Methods, ideas, or system contributions that make the work stand out.
3D Gaussian Splatting
Controllable Motion Generation
End-to-End Driving Safety Evaluation
Motion Diffusion
Photorealistic Scenario Synthesis
🔎 Similar Papers
No similar papers found.
