Problem
Research questions and friction points this paper is trying to address.
SAR-to-EO translation
foundation model
generalization
cross-domain adaptation
GeoSET: Generalist Foundation Model for SAR-to-EO Image Translation
🤖 AI Summary
This study addresses the limitation of existing SAR-to-EO image translation methods, which are typically confined to single datasets and struggle with multi-sensor, multi-resolution scenarios. To overcome this, we propose the first general-purpose foundation model for SAR-to-EO translation. Specifically, we design a speckle-noise-resistant encoder to extract robust features, pre-train a conditional generative parent model through large-scale data curation, and incorporate LoRA for parameter-efficient cross-domain adaptation. The proposed model achieves state-of-the-art performance in terms of FID and DISTS metrics across six benchmarks. Notably, it enables rapid transfer to novel domains by updating merely 0.6% of its parameters. This work provides an efficient and versatile solution for cross-modal remote sensing image generation.
📝 Abstract
Paired synthetic aperture radar (SAR) and electro-optical (EO) imagery is increasingly available across sensors, resolutions, and geographic regions. Yet existing SAR-to-EO image translation (SET) methods are typically trained on a single, limited-scale dataset, producing models specialized to particular sensing conditions. We introduce GeoSET, the first generalist model for SET, built around a single pretrained parent that is adapted to downstream datasets under a common protocol. We curate over 3 million high-quality SAR--EO pairs from a collection of more than 10 million SAR observations, spanning diverse sensors, spatial resolutions, and ground sampling distances. To bridge the modality gap between SAR observations and a pretrained image generator, we develop a speckle-robust SAR encoder and pretrain the conditional generator on this heterogeneous corpus. The resulting parent supports efficient adaptation across downstream datasets through low-rank adaptation (LoRA), updating only 0.60% of the generator parameters and requiring approximately one hour per dataset. Across six downstream benchmarks, GeoSET achieves state-of-the-art results in FID and DISTS with full fine-tuning or LoRA, demonstrating effective transfer across heterogeneous SAR-EO domains.
Innovation
Methods, ideas, or system contributions that make the work stand out.
SAR-to-EO translation
foundation model
speckle-robust encoder
low-rank adaptation
domain generalization
🔎 Similar Papers
No similar papers found.
