RIM: A Retrieval-In-Matching Framework for Cross-Domain Global Visual Localization of UAVs

2026-07-22

AI Summary

This work addresses the challenges of cross-domain appearance and viewpoint shifts in global visual localization for unmanned aerial vehicles, arising from temporal and platform discrepancies between remote sensing reference maps and onboard imagery. To tackle this, the authors propose a Retrieval-In-Matching (RIM) framework that leverages a DINOv2-B backbone and integrates a two-stage cross-domain fine-tuning strategy with a distillation-based local descriptor decoder. The method jointly performs global retrieval and local matching in a single forward pass by reusing DINOv2 token features and shallow VGG19 detail streams, eliminating the need for additional backbone networks. Evaluated zero-shot on the EPFL Urbanscape and Chang’an Park datasets, RIM improves Recall@1 by 8.55/13.77 and 4.45/8.94 percentage points at 25/50 meters, respectively, while achieving end-to-end localization in just 67.9 ms—1.8× faster than the strongest sparse matching baseline and over 40× faster than RoMa.

Abstract

Global visual localization of unmanned aerial vehicles (UAVs) using remote-sensing reference maps has attracted increasing attention. However, acquisition-time and imaging-platform differences between UAV and reference imagery induce substantial cross-domain appearance and viewpoint shifts, challenging robust six-degree-of-freedom (6-DoF) pose estimation. We address these shifts by sampling UAV-viewpoint reference views from Google 3D Tiles across locations, altitudes, and orientations. A two-stage cross-domain fine-tuning recipe adapts SALAD using pose-near positives and geographically distant hard negatives, while local geometric consistency re-ranks the Top-K candidates. We further propose Retrieval-In-Matching (RIM), which freezes the adapted DINOv2-B retriever and distils a local-descriptor decoder that reuses its token field alongside a shallow VGG19 detail stream. One query-side DINOv2-B forward thus serves both SALAD retrieval and local description, eliminating a second foundation-model backbone while preserving retrieval descriptors by construction. We evaluate RIM zero-shot on the reconstructed EPFL Urbanscape and self-collected Chang'an Park datasets, both geographically disjoint from the training data. RIM outperforms ten recent retrieval baseline families. At 25/50 m under the full 3D distance metric, it improves Recall@1 over SALAD by 8.55/13.77 percentage points on EPFL and 4.45/8.94 points on Park. At Top-K=5, the complete measured localization query, including retrieval, candidate matching, and robust geometric verification, takes 67.9 ms end-to-end: 1.8 times faster than the strongest separate sparse-matching baseline and over 40 times faster than RoMa, while achieving comparable re-ranking accuracy. These results establish an efficient and deployable pipeline for UAV global visual localization in GNSS-challenged environments.

Problem

• cross-domain
• visual localization
• UAV
• 6-DoF pose estimation
• appearance shift

Innovation

• Retrieval-In-Matching
• cross-domain localization
• DINOv2 distillation
• global visual localization
• UAV pose estimation