🤖 AI Summary
Autonomous driving testing requires efficient generation of simulation scenario code, yet non-programming domain experts struggle with this task. Method: We propose the first dialogue-based code generation system tailored for driving scenario modeling, integrating lightweight instruction fine-tuning, multi-turn dialogue state tracking, and domain-specific syntactic constraints to reliably translate natural language into symbolic scenario programs. Contribution/Results: Our work is the first to empirically validate the critical role of interactive dialogue in synthesizing complex scenario code. Leveraging minimal labeled data, the system achieves high-precision program synthesis. Human-in-the-loop experiments demonstrate that dialogue-based generation improves success rate by 4.5× over single-turn generation, significantly enhancing modeling accuracy, controllability, and practical usability for autonomous driving scenario creation.
📝 Abstract
Cyber-physical systems like autonomous vehicles are tested in simulation before deployment, using domain-specific programs for scenario specification. To aid the testing of autonomous vehicles in simulation, we design a natural language interface, using an instruction-following large language model, to assist a non-coding domain expert in synthesising the desired scenarios and vehicle behaviours. We show that using it to convert utterances to the symbolic program is feasible, despite the very small training dataset. Human experiments show that dialogue is critical to successful simulation generation, leading to a 4.5 times higher success rate than a generation without engaging in extended conversation.