Visual Localization via Semantic Structures in Autonomous Photovoltaic Power Plant Inspection

2025-01-24

AI Summary

Precise localization remains challenging in drone-based thermal inspection of photovoltaic (PV) power plants. Method: This paper proposes a semantic-structure-aware end-to-end visual localization approach that achieves real-time, centimeter-level alignment between aerial imagery and a 3D PV plant model by leveraging geometric–semantic features—such as panel layout patterns and visual anchor points. The method integrates classical computer vision (edge/texture analysis), deep learning (YOLO- and U-Net–based variants), and collaborative segmentation, enhanced by geometric constraint matching, semantic alignment, online tracking, and a novel interpretable visual anchor initialization–global tracking association mechanism. Contributions/Results: Evaluated on a custom aerial dataset, the method achieves <0.3 m localization error (37% improvement over SOTA) at 25 fps. It is the first to systematically characterize the impact of 3D model accuracy: every 10 cm reduction in model error improves localization accuracy by 22%. The collaborative segmentation strategy demonstrates superior robustness.

Technology Category

• Computer Vision: Segmentation
• Intelligent Robots: Localization, Mapping, and Navigation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Systems and Infrastructure for Web, Mobile and WoT: Location- and context-aware Web and WoT applications and services

Abstract

Inspection systems utilizing unmanned aerial vehicles (UAVs) equipped with thermal cameras are increasingly popular for the maintenance of photovoltaic (PV) power plants. However, automation of the inspection task is a challenging problem as it requires precise navigation to capture images from optimal distances and viewing angles. This paper presents a novel localization pipeline that directly integrates PV module detection with UAV navigation, allowing precise positioning during inspection. Detections are used to identify the power plant structures in the image and associate these with the power plant model. We define visually recognizable anchor points for the initial association and use object tracking to discern global associations. We present three distinct methods for visual segmentation of PV modules based on traditional computer vision, deep learning, and their fusion, and we evaluate their performance in relation to the proposed localization pipeline. The presented methods were verified and evaluated using custom aerial inspection data sets, demonstrating their robustness and applicability for real-time navigation. Additionally, we evaluate the influence of the power plant model's precision on the localization methods.

Problem

• Solar Power Station Inspection
• Drone Precision Autonomy
• Model Accuracy Impact

Innovation

• Photovoltaic Panel Recognition
• Drone Integration
• Real-time Navigation