On-the-fly Feedback SfM: Online Explore-and-Exploit UAV Photogrammetry with Incremental Mesh Quality-Aware Indicator and Predictive Path Planning

2025-12-02

AI Summary

For time-critical applications such as disaster response, real-time UAV photogrammetry lacks online quality assessment and adaptive data acquisition. To address this, we propose a closed-loop incremental 3D reconstruction framework. Methodologically, it integrates online Structure-from-Motion (SfM) sparse reconstruction, incremental mesh generation, and quantitative mesh quality evaluation, establishing a feedback mechanism that jointly balances exploration (covering occluded or undersampled regions) and exploitation (refining existing reconstructions), while enabling predictive trajectory replanning. Our key contribution is the first integration of mesh-quality awareness into the flight control loop, enabling dynamic, real-time assessment of reconstruction quality and online optimization of flight trajectories. Experiments demonstrate that, under near-real-time constraints, the method significantly reduces redundant image capture and coverage gaps, thereby improving reconstruction completeness and surveying efficiency—advancing UAV photogrammetry toward intelligent, adaptive operation.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Computer Vision: 3D Computer Vision
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Compared with conventional offline UAV photogrammetry, real-time UAV photogrammetry is essential for time-critical geospatial applications such as disaster response and active digital-twin maintenance. However, most existing methods focus on processing captured images or sequential frames in real time, without explicitly evaluating the quality of the on-the-go 3D reconstruction or providing guided feedback to enhance image acquisition in the target area. This work presents On-the-fly Feedback SfM, an explore-and-exploit framework for real-time UAV photogrammetry, enabling iterative exploration of unseen regions and exploitation of already observed and reconstructed areas in near real time. Built upon SfM on-the-fly , the proposed method integrates three modules: (1) online incremental coarse-mesh generation for dynamically expanding sparse 3D point cloud; (2) online mesh quality assessment with actionable indicators; and (3) predictive path planning for on-the-fly trajectory refinement. Comprehensive experiments demonstrate that our method achieves in-situ reconstruction and evaluation in near real time while providing actionable feedback that markedly reduces coverage gaps and re-flight costs. Via the integration of data collection, processing, 3D reconstruction and assessment, and online feedback, our on the-fly feedback SfM could be an alternative for the transition from traditional passive working mode to a more intelligent and adaptive exploration workflow. Code is now available at https://github.com/IRIS-LAB-whu/OntheflySfMFeedback.

Problem

• Enables real-time UAV photogrammetry for time-critical applications like disaster response.
• Provides online quality assessment and feedback during 3D reconstruction to guide image acquisition.
• Reduces coverage gaps and re-flight costs through predictive path planning and iterative exploration.

Innovation

• Online incremental mesh generation for expanding sparse point clouds
• Real-time mesh quality assessment with actionable indicators
• Predictive path planning for on-the-fly trajectory refinement