DerainSplat: Feed-Forward Clean 3D Gaussian Splatting from Sparse Rainy Views

2026-08-03

AI Summary

This work addresses the challenge of reconstructing clean 3D scenes from sparse multi-view images captured under rainy conditions, where existing 3D Gaussian splatting methods fail due to severe degradation from rain streaks and haze. We propose the first feed-forward framework that directly recovers rain-free 3D Gaussian representations from a small set of rainy input views. Our approach introduces a large-scale synthetic multi-view deraining dataset, a four-stage weather synthesis pipeline, and a weather-support map to guide geometric and radiometric consistency optimization. Key components include weather factor prediction, cross-view cost volume modulation, depth-aligned appearance fusion, and Gaussian opacity attenuation, all enhanced by rainy-cycle-consistent re-rendering for improved robustness. Extensive experiments on RealEstate10K, ACID, Mip-NeRF360, and real-world rainy scenes demonstrate significant performance gains over state-of-the-art methods and strong cross-dataset generalization.

Abstract

Although image deraining has advanced substantially, existing methods mainly focus on 2D image restoration. As spatial intelligence applications such as embodied AI and autonomous driving continue to emerge, reconstructing clean 3D scenes from sparse rainy views in a feed-forward manner becomes increasingly important. Existing feed-forward 3D Gaussian Splatting (3DGS) methods often assume clean inputs and collapse under rainy conditions. To this end, we present \textbf{\textit{DerainSplat}}, a feed-forward framework that reconstructs clean 3D scenes from only a few rainy views. To support this task, we build a large-scale multi-view derain dataset through a four-stage synthesis pipeline that sequentially models overcast illumination, depth-dependent haze, rain streaks, and lens raindrops, producing privileged weather factors. We introduce a weather net that predicts the weather factors from rainy context and yields two support maps. Scene support modulates cross-view cost-volume matching, while radiance support drives depth-aligned appearance fusion to fill corrupted pixels. The derived geometry evidence further attenuates Gaussian opacity to reduce spurious structures. A rainy cycle consistency re-renders clean views using the predicted factors and aligns them with rainy inputs. Extensive experiments show that \textbf{\textit{DerainSplat}} outperforms existing methods on various datasets, including RealEstate10K, ACID, Mip-NeRF360, and real-world rainy scenes, with strong cross-dataset generalization.

Problem

• 3D scene reconstruction
• image deraining
• rainy views
• feed-forward
• 3D Gaussian Splatting

Innovation

• 3D Gaussian Splatting
• image deraining
• feed-forward reconstruction
• weather-aware modeling
• multi-view consistency