STHN: Deep Homography Estimation for UAV Thermal Geo-Localization With Satellite Imagery

2024-05-30

IEEE Robotics and Automation Letters

AI Summary

To address the challenge of precise nighttime UAV localization under GNSS-denied conditions, this paper proposes a robust thermal-imaging geolocation method. Existing thermal–satellite cross-modal matching suffers from geometric noise, scale discrepancies (up to 11%), and severe texture scarcity, leading to failure in low-visibility nighttime scenarios. To overcome these limitations, we introduce a coarse-to-fine deep homography estimation framework—the first to enable high-robustness visual geolocation between thermal infrared and satellite imagery within a 512-meter radius. Our approach innovatively integrates CNN-based end-to-end homography estimation, multi-scale feature alignment, and thermal–satellite cross-modal geometric modeling. Evaluated on real-world field data, the method significantly improves both localization accuracy and robustness, enabling GNSS-free, all-weather autonomous navigation for UAVs.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: Remote Sensing / Geospatial AI
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Location- and context-aware Web and WoT applications and services

Abstract

Accurate geo-localization of Unmanned Aerial Vehicles (UAVs) is crucial for outdoor applications including search and rescue operations, power line inspections, and environmental monitoring. The vulnerability of Global Navigation Satellite Systems (GNSS) signals to interference and spoofing necessitates the development of additional robust localization methods for autonomous navigation. Visual Geo-localization (VG), leveraging onboard cameras and reference satellite maps, offers a promising solution for absolute localization. Specifically, Thermal Geo-localization (TG), which relies on image-based matching between thermal imagery with satellite databases, stands out by utilizing infrared cameras for effective nighttime localization. However, the efficiency and effectiveness of current TG approaches, are hindered by dense sampling on satellite maps and geometric noises in thermal query images. To overcome these challenges, we introduce STHN, a novel UAV thermal geo-localization approach that employs a coarse-to-fine deep homography estimation method. This method attains reliable thermal geo-localization within a 512-meter radius of the UAV's last known location even with a challenging 11% size ratio between thermal and satellite images, despite the presence of indistinct textures and self-similar patterns. We further show how our research significantly enhances UAV thermal geo-localization performance and robustness against geometric noises under low-visibility conditions in the wild.

Problem

• Enabling UAV geo-localization using thermal and satellite imagery
• Overcoming geometric noise and texture challenges in thermal images
• Providing robust localization under low-visibility and nighttime conditions

Innovation

• Employs coarse-to-fine deep homography estimation method
• Achieves geo-localization within 512-meter radius reliably
• Handles challenging 11% size ratio between image types