TrafficSignBench: Rule-Centric Closed-Loop Evaluation of Traffic-Sign Compliance in Autonomous Driving

2026-09-29

AI Summary

This study addresses the limitation of existing autonomous driving evaluation metrics, which lack explicit measurement of traffic rule compliance, allowing planners to achieve high scores while harboring violation risks. To this end, we construct a closed-loop evaluation benchmark based on real-world maps and rule-guided scenario generation. We propose an automated detector encompassing 34 categories of traffic rules alongside a dataset of 29,000 samples, enabling interpretable testing of traffic sign compliance. Furthermore, we transform planners into rule-compliant experts through explicit constraints. Our research reveals critical compliance deficiencies in prevailing mainstream planners and generates high-quality oracle trajectories, providing essential support for enhancing overall system safety.

Abstract

Autonomous driving planners are typically evaluated using aggregate metrics such as driving score, destination rate, and collision rate, which do not explicitly measure compliance with traffic rules. As a result, planners can achieve high benchmark scores while still exhibiting unsafe or illegal behaviors, limiting their applicability to real-world deployment. To address this gap, we introduce TrafficSignBench, a large-scale, traffic sign-centric benchmark for systematic and interpretable evaluation of traffic-rule compliance in autonomous driving. Our framework combines real-map-based simulation for realistic road layouts with rule-targeted procedural scenario generation for scalable and balanced coverage of underrepresented rules. We implement traffic rules corresponding to 34 traffic signs, each equipped with an automatic rule checker for detecting violations during closed-loop execution. This design yields 29,000 diverse road scenes and 29 distinct testing scenario types, enabling controlled evaluation of rule-specific planner behavior. We construct 5,800 testing scenes and demonstrate that current autonomous driving planners can exhibit poor traffic-rule compliance despite strong performance on standard evaluation metrics. To address this limitation, we transform existing planners into rule-compliant trajectory experts via explicit traffic-sign constraints, enabling scalable generation of high-quality oracle trajectories for fine-tuning.

Problem

• autonomous driving
• traffic sign compliance
• planner evaluation
• closed-loop benchmark

Innovation

• TrafficSignBench
• closed-loop evaluation
• traffic-rule compliance
• procedural scenario generation
• rule-compliant trajectory