Boosting Robustness for All-Weather Self-Supervised Depth Estimation in Autonomous Driving

2026-07-23

AI Summary

This work addresses the challenges of self-supervised monocular depth estimation under adverse weather conditions, where sensor degradation distorts pixel correspondences and sparse radar points in perspective view hinder effective fusion. To overcome these issues, the authors propose a self-training framework leveraging unpaired real-world all-weather data. The approach introduces an uncertainty-aware multi-teacher knowledge distillation mechanism to enhance model robustness and a novel POV-BEV radar-camera fusion strategy that bridges perspective and bird’s-eye views via camera ray constraints, enabling denser and more effective utilization of radar information. Evaluated on all-weather datasets, the method significantly outperforms existing approaches, achieving notably improved accuracy and robustness in depth estimation under rain, fog, and other challenging conditions.

Abstract

Self-supervised depth estimation is challenging for safe autonomous driving under various adverse weather conditions due to sensor perception degradation. These challenges arise from two main aspects. Firstly, adverse conditions can distort pixel correspondences and violate the assumptions embedded in the self-supervised loss function, leading to erroneous depth predictions. Secondly, while radar is a widely adopted sensor in adverse weather conditions, the sparse distribution of radar points in the Point of View (POV) poses challenges for self-supervised fusion. To address these issues, we introduce a novel self-training pipeline using unpaired real all-weather data through multi-teacher distillation and robust radar fusion. We propose the Uncertainty-Aware Multi-Teacher Distillation method to generate diverse teacher models with different adverse condition inputs, and then employ uncertainty modeling to weigh the knowledge distillation loss. Additionally, we design the POV-BEV Radar Fusion approach, which leverages camera-pixel ray constraints to establish connections between the camera's Point of View (POV) and the radar's Bird's-Eye View (BEV). This approach enables the utilization of denser radar points, effectively capturing the complementary perspectives of both POV and BEV. Extensive quantitative and qualitative experiments demonstrate the robustness of our proposed method on all-weather datasets, achieving state-of-the-art performance. Our code and models are available at https://github.com/MICLAB-BUPT/RobustDepth.

Problem

• self-supervised depth estimation
• adverse weather conditions
• sensor fusion
• radar sparsity
• pixel correspondence

Innovation

• Uncertainty-Aware Multi-Teacher Distillation
• POV-BEV Radar Fusion
• Self-Supervised Depth Estimation
• All-Weather Perception
• Unpaired Data Training