Fusing Satellite Imagery and Planimetric Maps for Cross-View Localization

2026-06-08

AI Summary

Existing cross-view localization methods predominantly rely on a single aerial modality—such as satellite imagery or top-down maps—struggling to simultaneously capture fine-grained texture and large-scale structural information, which limits localization accuracy. This work proposes a novel end-to-end fusion framework that, for the first time, integrates a cross-modal conditioning mechanism and a block-level fusion strategy within the encoder to adaptively leverage the complementary strengths of satellite images (rich in textural detail) and map data (structured with semantic layout). By enabling joint representation learning from heterogeneous geospatial sources, the method achieves state-of-the-art performance on standard benchmarks, reducing the average localization error by 30.13% and substantially surpassing the limitations inherent to single-modality approaches.

Abstract

Current cross-view localization methods predominantly rely on satellite imagery as the aerial modality. Although recent work explores planimetric maps (e.g., OpenStreetMap tiles), these approaches often lag in performance. Yet both modalities are widely available and possess complementary properties. Satellite images are closer to ground-level camera imagery, offering finer detail, whereas planimetric maps contain annotated objects (e.g., streetlamps) and remain informative in areas where the ground is occluded, such as by foliage. Despite this, only one prior work provides an end-to-end method to fuse the two modalities, and it does not demonstrate their potential within state-of-the-art methods. To combine the strengths of both modalities, we propose a new fusion module that augments standard encoders and demonstrates that integrating satellite imagery with planimetric maps improves state-of-the-art single-modality methods. The module comprises (i) cross-modal conditioning, which processes each modality's encoding with awareness of the other, and (ii) a patch-level fusion rule that controls the granularity of information exchange. We achieve state-of-the-art results, reducing the mean localization error by 30.13\%. Qualitatively, the fusion adaptively selects the more informative modality, improving overall accuracy.

Problem

• cross-view localization
• satellite imagery
• planimetric maps
• multimodal fusion
• geospatial data

Innovation

• cross-view localization
• satellite imagery
• planimetric maps
• multimodal fusion
• patch-level fusion