Learning from synthetic photorealistic raindrop for single image raindrop removal

2026-09-29

AI Summary

This study addresses the severe degradation of visual systems caused by lens-attached raindrops and the critical scarcity of real-world training data in existing research. To overcome these limitations, this work integrates physical modeling with deep learning, leveraging physics-based rendering to construct the first high-fidelity synthetic raindrop dataset. Building upon this dataset, we design a refraction- and blur-aware network for joint detection and removal, enabling end-to-end single-image raindrop restoration. By transcending the constraints of conventional techniques in modeling complex raindrop appearances, the proposed method achieves state-of-the-art performance across multiple benchmarks. Ultimately, this research provides an effective solution for image restoration in real-world scenarios, significantly advancing the robustness of vision systems operating under adverse weather conditions.

Abstract

Raindrops adhered to camera lens or windshield are inevitable in rainy scenes and can become an issue for many computer vision systems such as autonomous driving. Because raindrop appearance is affected by too many parameters, therefore it is unlikely to find an effective model based solution. Learning based methods are also problematic, because traditional learning method cannot properly model the complex appearance. Whereas deep learning method lacks sufficiently large and realistic training data. To solve it, in our work, we propose the first photo-realistic dataset of synthetic adherent raindrops for training. The rendering is physics based with consideration of the water dynamic, geometric and photometry. The dataset contains various types of rainy scenes and particularly the rainy driving scenes. Based on the modeling of raindrop imagery, we introduce a detection network which has the awareness of the raindrop refraction as well as its blurring. Based on that, we propose the removal network that can well recover the image structure. Rigorous experiments demonstrate the state-of-the-art performance of our proposed framework.

Problem

• raindrop removal
• synthetic dataset
• single image
• autonomous driving
• deep learning

Innovation

• raindrop removal
• synthetic dataset
• physics-based rendering
• detection network
• refraction awareness