Problem
Research questions and friction points this paper is trying to address.
closed-loop safety
end-to-end driving policies
photorealistic environments
safety evaluation
NavSafe-$\infty$: Benchmarking Closed-Loop Driving Safety in Photorealistic Environments
🤖 AI Summary
研究引入NavSafe-∞，在280个场景中评估端到端驾驶策略的闭环安全性，发现开环性能不能可靠地转化为闭环安全，并分析了两种常见补救措施的效果。
📝 Abstract
End-to-end (E2E) driving policies have progressed rapidly on open-loop (OL) benchmarks, yet OL evaluation cannot reveal whether a policy withstands compounding errors, recovers from failures, or interacts safely with surrounding actors. We introduce NavSafe-$\infty$, a photorealistic closed-loop (CL) benchmark of 280 scenarios spanning 28 event types, each with success and failure criteria defined within a structured traffic-safety taxonomy, which yields category-level capability scores for Traffic Crashes, Vulnerable Road User Crashes, Traffic Violations, and Traffic Incidents. Evaluating 20 E2E policies, we find that OL gains do not reliably transfer to CL safety. Analyzing two common remedies further shows that passive demonstration perturbation helps mainly when CL rollouts stay near its perturbed training states, and that OL reinforcement-learning fine-tuning exhibits reward hacking by trading safety margin for ego progress, which CL feedback amplifies into compounding safety-critical errors. Together, these results demonstrate the blind spot of OL benchmarks indicating CL safety success. The benchmark and an extensible toolbox for customizable event curation and policy diagnosis will be open-sourced and maintained to facilitate future research.
Innovation
Methods, ideas, or system contributions that make the work stand out.
photorealistic closed-loop benchmark
end-to-end driving policies
traffic-safety taxonomy
compounding errors
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
