Assessing the Benefits of Combining Advanced Deep Learning Techniques for Post-Disaster Building Damage Assessment from UAV Imagery

2026-08-03

AI Summary

This study addresses the limitations of existing post-disaster building damage assessment methods, which typically rely on large amounts of labeled data, exhibit poor cross-regional generalization, and offer limited task adaptability. To overcome these challenges, the authors propose a decoupled hybrid framework that first leverages high-precision computer vision models—such as Grounding DINO—to accurately localize buildings, followed by a large vision-language model (LVLM) for damage classification and contextual reasoning. This approach uniquely integrates the precise detection capabilities of computer vision with the semantic reasoning strengths of LVLMs, achieving significantly improved assessment performance with only minimal annotated data. Evaluated on real-world disaster datasets RescueNet and FloodNet, the method outperforms single-model baselines by up to 2.1 R² points and accurately quantifies the counts of undamaged, partially damaged, and completely destroyed buildings, demonstrating strong generalization and practical utility.

Abstract

Rapid and accurate post-disaster building damage assessment is essential, yet remains a challenging task. Unmanned Aerial Vehicle (UAV) imagery offers a timely and high-resolution view of affected areas, but existing Computer Vision (CV) models often demand large annotated datasets, generalize poorly across geographic regions and their assessment policies, and are confined to the specific tasks they were trained for. Large Vision-Language Models (LVLMs) offer a promising alternative through their strong reasoning and generalization capabilities, but fall short on precise, low-level perception tasks such as object detection and accurate bounding box generation. Furthermore, they often require a substantial amount of data for effective fine-tuning on domain-specific tasks. In this paper, we propose a hybrid framework that decouples detection from damage assessment, combining the precision of CV models with the reasoning power of LVLMs. A CV model first detects buildings and generates bounding boxes on the image that are then passed to an LVLM for damage classification and contextual interpretation. We evaluated our framework on two real-world benchmarks: RescueNet and FloodNet. In particular, the best combination under this framework accurately counts intact, partially damaged and completely destroyed buildings, surpassing isolated baselines by up to 2.1 R^2 points, while requiring only limited annotated data for the detection stage. Beyond reporting aggregate gains, we provide a detailed analysis of failure scenarios and edge cases, offering practical insights for practitioners and concrete directions for future work. Our source code and data are publicly available to the research community via the following repository: https://github.com/ungquanghuy-kddi/VLM_GDINO.git

Problem

• post-disaster building damage assessment
• UAV imagery
• computer vision
• large vision-language models
• object detection

Innovation

• hybrid framework
• Large Vision-Language Models
• building damage assessment
• UAV imagery
• few-shot detection