Hyperlocal disaster damage assessment using bi-temporal street-view imagery and pre-trained vision models

2025-04-12

AI Summary

This work addresses the insufficient fine-grained discrimination accuracy in ultra-localized disaster damage assessment. We propose a joint analysis method leveraging bitemporal street-view imagery (pre- and post-disaster) and pretrained vision models. Our approach introduces a novel dual-channel feature fusion framework: using pre-disaster imagery as an undamaged reference, it establishes a paired-image joint inference mechanism to achieve cross-temporal semantic alignment and enhance damage-sensitive features. We adopt Swin Transformer and ConvNeXt as backbone architectures, integrating transfer learning with temporal street-view analysis. Experiments demonstrate that our method significantly improves building- and street-level damage classification accuracy from 66.14% (Swin baseline) to 77.11%. This advancement substantially enhances rapid, operationally actionable ultra-local assessment capabilities post-disaster, offering a new paradigm for resilient urban planning and emergency decision-making.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Street-view images offer unique advantages for disaster damage estimation as they capture impacts from a visual perspective and provide detailed, on-the-ground insights. Despite several investigations attempting to analyze street-view images for damage estimation, they mainly focus on post-disaster images. The potential of time-series street-view images remains underexplored. Pre-disaster images provide valuable benchmarks for accurate damage estimations at building and street levels. These images could aid annotators in objectively labeling post-disaster impacts, improving the reliability of labeled data sets for model training, and potentially enhancing the model performance in damage evaluation. The goal of this study is to estimate hyperlocal, on-the-ground disaster damages using bi-temporal street-view images and advanced pre-trained vision models. Street-view images before and after 2024 Hurricane Milton in Horseshoe Beach, Florida, were collected for experiments. The objectives are: (1) to assess the performance gains of incorporating pre-disaster street-view images as a no-damage category in fine-tuning pre-trained models, including Swin Transformer and ConvNeXt, for damage level classification; (2) to design and evaluate a dual-channel algorithm that reads pair-wise pre- and post-disaster street-view images for hyperlocal damage assessment. The results indicate that incorporating pre-disaster street-view images and employing a dual-channel processing framework can significantly enhance damage assessment accuracy. The accuracy improves from 66.14% with the Swin Transformer baseline to 77.11% with the dual-channel Feature-Fusion ConvNeXt model. This research enables rapid, operational damage assessments at hyperlocal spatial resolutions, providing valuable insights to support effective decision-making in disaster management and resilience planning.

Problem

• Assessing hyperlocal disaster damage using bi-temporal street-view imagery
• Improving damage estimation accuracy with pre- and post-disaster image pairs
• Enhancing vision models for street-level disaster impact classification

Innovation

• Bi-temporal street-view imagery for damage assessment
• Pre-trained vision models like Swin Transformer
• Dual-channel algorithm for pairwise image analysis