DriveBLIP2: Attention-Guided Explanation Generation for Complex Driving Scenarios

2025-06-24

AI Summary

To address the limited interpretability of vision-language models in complex, multi-object autonomous driving scenarios, this paper proposes a multimodal interpretability framework built upon BLIP-2-OPT. The core innovation is an attention map generator that explicitly guides the model to focus on object regions critical for driving decisions. The framework integrates frame-level key-object localization with cross-modal alignment, enabling synergistic visual–linguistic reasoning. Experiments on the DRAMA dataset demonstrate substantial improvements in explanation accuracy and contextual relevance: the proposed method outperforms baseline models across all standard metrics—BLEU, ROUGE, CIDEr, and SPICE—validating its effectiveness in enhancing model interpretability within dynamic driving environments.

Technology Category

• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

This paper introduces a new framework, DriveBLIP2, built upon the BLIP2-OPT architecture, to generate accurate and contextually relevant explanations for emerging driving scenarios. While existing vision-language models perform well in general tasks, they encounter difficulties in understanding complex, multi-object environments, particularly in real-time applications such as autonomous driving, where the rapid identification of key objects is crucial. To address this limitation, an Attention Map Generator is proposed to highlight significant objects relevant to driving decisions within critical video frames. By directing the model's focus to these key regions, the generated attention map helps produce clear and relevant explanations, enabling drivers to better understand the vehicle's decision-making process in critical situations. Evaluations on the DRAMA dataset reveal significant improvements in explanation quality, as indicated by higher BLEU, ROUGE, CIDEr, and SPICE scores compared to baseline models. These findings underscore the potential of targeted attention mechanisms in vision-language models for enhancing explainability in real-time autonomous driving.

Problem

• Generating accurate explanations for complex driving scenarios
• Improving understanding of multi-object environments in real-time
• Enhancing explainability in autonomous driving decision-making

Innovation

• DriveBLIP2 enhances BLIP2-OPT for driving scenarios
• Attention Map Generator highlights key driving objects
• Improves explanation quality with targeted attention mechanisms