ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness

2026-07-26

AI Summary

Existing vision-language benchmarks struggle to evaluate multimodal perception capabilities of autonomous driving systems under real-world adverse weather conditions, primarily due to the lack of modeling environmental observability degradation. To address this gap, this work introduces ObsDriveBench—the first fine-grained evaluation benchmark specifically designed to assess multimodal inconsistency and unreliability in adverse weather—by incorporating the novel concept of "observability awareness." The benchmark leverages synchronized camera, LiDAR, and radar data, observability-aware meta-annotations, and a capability-oriented multiple-choice question task to enable fine-grained diagnosis across three dimensions: observability perception, spatial reliability, and risk-aware decision-making. Building upon this benchmark, we propose the ObsDrive model, which combines supervised fine-tuning under normal weather with reinforcement learning in adverse conditions, achieving significantly improved robustness on 14K training and 13K test questions and revealing critical performance bottlenecks of current models under degraded observability.

Abstract

Autonomous driving under adverse weather remains a critical challenge, yet existing vision-language benchmarks mainly evaluate under standard conditions, synthetic corruptions, or single modality. As a result, it remains unclear how vision-language models behave under real-world adverse weather with multi-modal inputs. We argue that a key difficulty lies in degraded environmental observability: under fog, rain, snow, and low illumination, multi-modal observations become unreliable and cross-modally inconsistent, posing challenges to scene understanding, and subsequent decision-making. To study this, we introduce \textbf{ObsDriveBench}, a real-world multi-modal benchmark for adverse-weather autonomous driving. Our benchmark is designed with three capability dimensions: \textbf{observability awareness}, \textbf{spatial reliability}, and \textbf{risk-aware decision-making}, enabling fine-grained diagnosis of model behavior under degraded observations. We construct the benchmark through observability meta-annotation, scene description, and capability oriented multiple-choice tasks over synchronized camera, LiDAR, and radar inputs, forming a benchmark with over 14k training and 13k test questions. Experiments reveal consistent performance degradation of existing vision-language models. We further introduce \textbf{ObsDrive} model with normal-weather supervised fine-tuning and adverse-weather reinforcement learning, improving robustness across all three capabilities. The dataset and evaluation code will be released at \href{https://github.com/russellyq/ObsDriveBench}{\texttt{ObsDriveBench}}.

Problem

• adverse weather
• multimodal understanding
• observability awareness
• autonomous driving
• cross-modal inconsistency

Innovation

• observability awareness
• multimodal benchmark
• adverse weather driving
• risk-aware decision-making
• reinforcement learning