DispatchRAG: Grounding Emergency Dispatch Decisions in Real-World Protocols from Traffic Accident Video

2026-07-25

AI Summary

This study addresses the critical limitation of existing vision-language models (VLMs) in adhering to real-world traffic accident emergency protocols, which often leads to misjudgments of incident severity and erroneous emergency service dispatch. To bridge this gap, the work proposes a retrieval-augmented generation (RAG)-based framework that integrates official traffic accident response protocols—specifically those from Japan—into a vision-language reasoning pipeline. By retrieving relevant protocol excerpts via RAG and constraining inference through the combined use of VLMs and large language models, the system enables protocol-compliant accident assessment and dispatch decisions. The authors also introduce the first benchmark dataset aligned with official protocols for accident dispatch evaluation, on which their method significantly outperforms existing VLM baselines across diverse accident scenarios, demonstrating high accuracy and offering a viable solution for automated accident reporting in autonomous vehicles.

Abstract

Assessing the severity of a traffic accident scenario is important to decide which emergency service to dispatch. Missing an ambulance dispatch on a pedestrian accident is a fatal issue that can lead to death. Recently, Vision-Language Models (VLMs) have been a promising tool for accident reasoning, yet many VLMs are not grounded in real-life accident response protocols, making them not usable in accident severity assessment off-the-shelf. We introduced DispatchRAG, an accident assessor and dispatcher framework grounded in real-life Japanese traffic-accident response protocols, designed to enhance VLMs to generate an appropriate emergency response during an emergency scenario. Utilizing a RAG-based retrieval mechanism to retrieve the most relevant accident protocol and an LLM-powered reasoner to suggest the most proper response. To support evaluation, we introduce Accident Dispatch Dataset, a comprehensive dataset of accident assessment and emergency response according to Japanese accident response protocols adapted from the MM-AU dataset. We validate our framework on the Accident Dispatch Dataset, showing strong performance across various accident scenarios compared to the baseline VLM, pointing toward integration in autonomous vehicles that can automatically report both their own and nearby accidents.

Problem

• emergency dispatch
• traffic accident severity
• vision-language models
• real-world protocols
• accident response

Innovation

• DispatchRAG
• Retrieval-Augmented Generation (RAG)
• Vision-Language Models (VLMs)
• Emergency Dispatch Protocols
• Accident Severity Assessment