When Cloud Removal Meets Diffusion Model in Remote Sensing

2025-04-21

AI Summary

Cloud occlusion in remote sensing imagery causes critical loss of underlying surface information, severely hindering downstream applications. To address this, we propose DC4CR—the first prompt-driven multimodal diffusion framework for cloud removal, capable of selectively removing both thin and thick clouds without requiring pre-generated cloud masks. Methodologically, DC4CR innovatively integrates prompt-based control mechanisms, Low-Rank Adaptation (LoRA), subject-driven generation, and grouped learning strategies, substantially enhancing few-shot generalization capability and inference efficiency; its modular architecture enables plug-and-play deployment. Evaluated on the RICE and CUHK-CR benchmarks, DC4CR achieves state-of-the-art performance, demonstrating superior reconstruction accuracy and robustness under complex cloud conditions compared to existing methods, thereby exhibiting strong practical potential.

Technology Category

Computer Vision: Diffusion Models for VisionMachine Learning: Multimodal LearningIntelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAGEconomics, Online Markets and Human Computation: LLM based quality controls for crowd workUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Cloud occlusion significantly hinders remote sensing applications by obstructing surface information and complicating analysis. To address this, we propose DC4CR (Diffusion Control for Cloud Removal), a novel multimodal diffusion-based framework for cloud removal in remote sensing imagery. Our method introduces prompt-driven control, allowing selective removal of thin and thick clouds without relying on pre-generated cloud masks, thereby enhancing preprocessing efficiency and model adaptability. Additionally, we integrate low-rank adaptation for computational efficiency, subject-driven generation for improved generalization, and grouped learning to enhance performance on small datasets. Designed as a plug-and-play module, DC4CR seamlessly integrates into existing cloud removal models, providing a scalable and robust solution. Extensive experiments on the RICE and CUHK-CR datasets demonstrate state-of-the-art performance, achieving superior cloud removal across diverse conditions. This work presents a practical and efficient approach for remote sensing image processing with broad real-world applications.

Problem

• Removing cloud occlusion in remote sensing imagery
• Enhancing preprocessing efficiency without cloud masks
• Improving model adaptability and computational efficiency

Innovation

• Prompt-driven control for selective cloud removal
• Low-rank adaptation for computational efficiency
• Plug-and-play module for seamless integration