Multi-Sensor Alignment for Weather Simulations

2026-07-28

AI Summary

Current autonomous driving perception systems suffer degraded performance under adverse weather conditions, yet existing multi-sensor weather simulation datasets lack consistent alignment between weather intensity and particle placement, leading to overly optimistic model evaluations. This work proposes ReDAM to align fog intensity and Unified-weather-edit to align particle positions in rain and snow, achieving— for the first time—the joint alignment of these two critical weather characteristics in multi-sensor simulation. The effectiveness of the alignment is rigorously validated through statistical tests and geometric consistency checks. When integrated with fine-tuned sensor fusion models, the proposed approach significantly enhances the robustness of 3D object detection in adverse weather. Furthermore, it demonstrates that misaligned synthetic data inflates performance estimates, thereby underscoring the necessity and efficacy of physically consistent weather alignment.

Abstract

Perception tasks for autonomous vehicles need to work satisfactorily in adverse weather conditions. Due to lack of real-world weather datasets, weather simulations are a promising alternative. To ensure simulations closely mirror real-world weather data, it's crucial that they represent the same weather characteristics, including severity and particle positioning, across different sensors. To achieve this, we propose the Reference Dataset Alignment Method (ReDAM) for weather intensity alignment in fog and Unified-weather-edit (inspired by Weather-edit[1]) for particle positioning alignment in rain and snow. We validate both alignment methods using statistical and geometrical tests, respectively. We find that 3D detection models for non-aligned versions tend to be overly optimistic as compared to aligned versions. We also show the aligned-multi-sensor simulation's effectiveness for achieving robustness for 3D object detection task by finetuning existing sensor fusion models on it.

Problem

• multi-sensor alignment
• weather simulation
• adverse weather conditions
• 3D object detection
• sensor fusion

Innovation

• multi-sensor alignment
• weather simulation
• 3D object detection
• sensor fusion
• adverse weather robustness