Cross-View Image Set Geo-Localization

2024-12-25

AI Summary

This work addresses the underutilization of multi-view ground-level imagery in cross-view geo-localization (CVGL) by proposing a novel “image set as query” paradigm. To support this, we introduce SetVL-480K—a large-scale benchmark comprising 480K satellite–multi-view ground image pairs, with each satellite image associated with an average of 40 ground views. Methodologically, we propose a Similarity-Guided Feature Fusion (SFF) module that adaptively fuses multi-view features without requiring content priors, and an Image-wise Attribute Learning (IAL) module for fine-grained geographic modeling. Our unified framework jointly optimizes deep feature extraction, multi-view set representation, and geographic attribute learning. Evaluated on SetVL-480K, our method achieves over 22% improvement in localization accuracy. It also consistently outperforms state-of-the-art methods on SeqGeo and KITTI-CVL, while natively supporting single-image, sequential, and image-set inputs.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Cross-view geo-localization (CVGL) has been widely applied in fields such as robotic navigation and augmented reality. Existing approaches primarily use single images or fixed-view image sequences as queries, which limits perspective diversity. In contrast, when humans determine their location visually, they typically move around to gather multiple perspectives. This behavior suggests that integrating diverse visual cues can improve geo-localization reliability. Therefore, we propose a novel task: Cross-View Image Set Geo-Localization (Set-CVGL), which gathers multiple images with diverse perspectives as a query set for localization. To support this task, we introduce SetVL-480K, a benchmark comprising 480,000 ground images captured worldwide and their corresponding satellite images, with each satellite image corresponds to an average of 40 ground images from varied perspectives and locations. Furthermore, we propose FlexGeo, a flexible method designed for Set-CVGL that can also adapt to single-image and image-sequence inputs. FlexGeo includes two key modules: the Similarity-guided Feature Fuser (SFF), which adaptively fuses image features without prior content dependency, and the Individual-level Attributes Learner (IAL), leveraging geo-attributes of each image for comprehensive scene perception. FlexGeo consistently outperforms existing methods on SetVL-480K and two public datasets, SeqGeo and KITTI-CVL, achieving a localization accuracy improvement of over 22% on SetVL-480K.

Problem

• Cross-View Geo-Localization
• Accuracy Improvement
• Multi-Angle Imagery

Innovation

• Set-CVGL
• FlexGeo
• Multi-perspective Imaging