Panopticon: Advancing Any-Sensor Foundation Models for Earth Observation

2025-03-13

AI Summary

Earth observation (EO) data exhibit multi-source heterogeneity (e.g., optical/SAR) and sensor diversity, severely limiting model generalizability across platforms. Method: We propose the first foundation model for arbitrary-sensor EO data. Our approach treats co-located multi-source imagery as natural augmentation; introduces channel sub-sampling to enhance spectral diversity; and employs cross-channel attention for band-agnostic, flexible embedding. Built upon the DINOv2 framework, it integrates wavelength encoding, SAR modality encoding, and self-supervised contrastive learning. Contribution/Results: Our model breaks the fixed-sensor paradigm, enabling unified representation and joint modeling of optical and SAR data. It achieves state-of-the-art performance on GEO-Bench—significantly outperforming existing arbitrary-sensor models and single-sensor adaptation methods—and supports plug-and-play generalization across both legacy and emerging satellite platforms.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Earth observation (EO) data features diverse sensing platforms with varying spectral bands, spatial resolutions, and sensing modalities. While most prior work has constrained inputs to fixed sensors, a new class of any-sensor foundation models able to process arbitrary sensors has recently emerged. Contributing to this line of work, we propose Panopticon, an any-sensor foundation model built on the DINOv2 framework. We extend DINOv2 by (1) treating images of the same geolocation across sensors as natural augmentations, (2) subsampling channels to diversify spectral input, and (3) adding a cross attention over channels as a flexible patch embedding mechanism. By encoding the wavelength and modes of optical and synthetic aperture radar sensors, respectively, Panopticon can effectively process any combination of arbitrary channels. In extensive evaluations, we achieve state-of-the-art performance on GEO-Bench, especially on the widely-used Sentinel-1 and Sentinel-2 sensors, while out-competing other any-sensor models, as well as domain adapted fixed-sensor models on unique sensor configurations. Panopticon enables immediate generalization to both existing and future satellite platforms, advancing sensor-agnostic EO.

Problem

• Develops a sensor-agnostic model for Earth observation data.
• Enhances DINOv2 to process diverse spectral and spatial inputs.
• Achieves state-of-the-art performance on GEO-Bench benchmarks.

Innovation

• Extends DINOv2 with natural augmentations across sensors
• Subsamples channels to diversify spectral input
• Adds cross attention for flexible patch embedding