SLED: Scalable Location Encoding via Distillation

2026-08-06

AI Summary

This work addresses the limitations of existing positional encoding methods, which rely on computationally expensive CLIP-style frameworks requiring massive batch sizes, suffering from false-negative interference, and struggling to scale to multimodal remote sensing data. The authors propose a lightweight, knowledge distillation–based positional encoding framework that uses geographic coordinates as a unified binding modality, enabling flexible fusion of arbitrary Earth observation data—such as Sentinel-1/2 and Landsat—without requiring spatiotemporal alignment during pretraining. Introducing a novel modular distillation mechanism, the method eliminates dependence on large batch training and precise image registration, achieving efficient learning with a batch size as small as 128. It matches or surpasses state-of-the-art approaches across 19 human-centric benchmark tasks, demonstrating strong scalability and effective multimodal integration.

Abstract

The plethora of readily available geospatial data offers exciting opportunities to learn high quality representations of the planet, but the sheer size of the Earth Observations (EO), differing modalities, and different sensor types pose significant challenges in doing so. Location encoders have emerged as an efficient way of compressing EOs into location-specific embeddings. However, current state-of-the-art location encoders rely on computationally expensive CLIP-style frameworks that require large batch sizes in the 16K--32K range, suffer from false negative samples, and scale poorly with additional modalities. We introduce the Scalable Location Encoder via Distillation (SLED), a distillation-based location encoder that uses geospatial location as a binding modality to pretrain location encoders with any modality of geospatial data. The resulting location encoder framework is lightweight, modular, and can flexibly incorporate multiple modes, while eliminating the need for spatiotemporal coregistration of samples. SLED is performant with batch sizes as small as 128, enabling pretraining at a fraction of the runtime and compute costs of current state-of-the-art models. We demonstrate our approach by pretraining unimodal and multimodal SLED models on Sentinel-1, Sentinel-2, and Landsat imagery. We show that both unimodal and multimodal SLED models keep pace with or outperform existing approaches on a diverse set of 19 human-centric benchmark tasks and explore the benefits of using additional modes in pretraining.

Problem

• location encoding
• Earth Observations
• multimodal learning
• scalability
• computational efficiency

Innovation

• location encoding
• knowledge distillation
• multimodal learning
• geospatial representation
• scalable pretraining