GeoCR: Learning a Generalist Cloud Removal Prior from Heterogeneous Observations

2026-09-26

AI Summary

This study addresses the limited reusability of existing cloud removal methods across heterogeneous sensors, spectral bands, and observation configurations. To overcome this, we propose a unified framework that jointly models RGB and multispectral data via a shared latent interface. For the first time, a single network accommodates heterogeneous inputs by freezing a pretrained backbone and extending compact ports, enabling zero-shot inference without fine-tuning as well as efficient adaptation via LoRA. The approach further integrates flow transformers, SAR-guided conditioning, and multi-dataset joint pretraining to learn generalizable priors. Extensive experiments on multiple benchmarks demonstrate that our method achieves state-of-the-art FID and DISTS scores, significantly outperforming existing models and validating its effectiveness for cross-scenario reuse.

Abstract

Cloud removal methods are typically specialized to individual datasets and input configurations, limiting reuse across sensors, spectral bands, and observation settings. We introduce GeoCR, a generalist model that unifies RGB-only-based CR and multispectral-based CR from single- or multi-temporal cloudy observations, with optional SAR guidance, within a single network. To accommodate different spectral and sensing domains, compact input and output stems extend a pretrained RGB autoencoder while keeping its encoder and decoder trunks frozen. This shared latent interface enables a single flow transformer to jointly model clean RGB and non-RGB latents, conditioned on separate cloudy-observation streams and optional SAR tokens. Through joint pretraining on the training splits of ten datasets comprising 883,331 cloud-free target images, GeoCR learns a shared cloud removal prior across these heterogeneous configurations. The same pretrained checkpoint supports direct inference without dataset-specific fine-tuning and efficient adaptation through low-rank adaptation (LoRA). We evaluate GeoCR against general image restoration and cloud removal methods on test splits of the contributing datasets under full-band and RGB-only settings. GeoCR achieves the best FID and DISTS on full-band SEN12MS-CR and Sen2_MTC_New and RGB-only CUHK-CR2, outperforming existing models and demonstrating the effectiveness of a reusable generative model across diverse settings.

Problem

• Cloud Removal
• Generalist Model
• Heterogeneous Observations
• Image Restoration

Innovation

• Generalist Cloud Removal
• Flow Transformer
• Shared Latent Space
• Heterogeneous Observations
• Low-Rank Adaptation