SAREO-FM: Decoupled Semantic Supervision for SAR-EO Foundation Models

2026-10-06

AI Summary

This work addresses the inherent conflict in SAR and EO image fusion, where a single token stream struggles to simultaneously accommodate modality reconstruction and semantic understanding. To this end, we propose SAREO-FM, a framework featuring a novel dual-token-stream decoupled supervision mechanism. Specifically, masked reconstruction and semantic querying are allocated to independent token streams while maintaining interaction within a shared encoder to prevent role confusion. Furthermore, pretrained vision foundation models are leveraged to guide learnable semantic queries for enhanced complementary perception. Pretrained on million-scale data, SAREO-FM significantly improves single-modality transferability and complementary perception performance under multi-source joint observation.

Abstract

Synthetic aperture radar (SAR) and electro-optical (EO) imagery provide complementary observations: SAR enables day-and-night, weather-resilient sensing, whereas EO provides rich appearance and fine-grained semantic cues. We introduce SAREO-FM, which avoids forcing a single token stream to serve two distinct roles: modality tokens preserve how each sensor observes the scene through masked reconstruction, while learnable semantic queries capture what the scene contains under guidance from a pretrained vision foundation model (VFM). By jointly encoding these queries with SAR and EO tokens, the queries acquire modality-grounded semantic context, while the modality-token outputs remain the explicit targets of masked reconstruction. This design assigns semantic and reconstruction supervision to separate token streams while preserving their interaction within the shared encoder. Pretrained on the million-scale SAR-1M corpus, SAREO-FM achieves strong unimodal transfer for both SAR-only and EO-only inputs, while delivering substantial gains from joint SAR--EO observations on tasks that benefit from complementary sensing.

Problem

• Synthetic Aperture Radar
• Electro-Optical Imagery
• Foundation Model
• Multimodal Fusion
• Semantic Supervision

Innovation

• Decoupled Semantic Supervision
• Foundation Model
• SAR-EO Fusion
• Masked Reconstruction
• Vision Foundation Model