Instruct-ICL: Instruction-Guided In-Context Learning for Post-Disaster Damage Assessment

2026-05-11

AI Summary

This work addresses the sensitivity of multimodal large language models (MLLMs) to prompt phrasing and their unstable reasoning in post-disaster visual question answering (VQA). To mitigate these issues, the authors propose an instruction-guided in-context learning framework that leverages one MLLM to generate task-specific structured instructions as a chain-of-thought (CoT), which then guides another MLLM in assessing disaster damage. The approach integrates contextual examples at multiple granularities to enhance reasoning robustness. Experimental results on the FloodNet dataset demonstrate that the proposed method significantly outperforms zero-shot baselines, confirming the effectiveness of instruction-driven CoT in improving both accuracy and robustness for post-disaster VQA tasks.

Abstract

Rapid and accurate situational awareness is essential for effective response during natural disasters, where delays in analysis can significantly hinder decision-making. Training task-specific models for post-disaster assessment is often time-consuming and computationally expensive, making such approaches impractical in time-critical scenarios. Consequently, pretrained multimodal large language models (MLLMs) have emerged as a promising alternative for post-disaster visual question answering (VQA), a task that aims to answer structured questions about visual scenes by jointly reasoning over images and text. While these models demonstrate strong multimodal reasoning capabilities, their responses can be sensitive to prompt formulation, which can limit their reliability in real-world disaster assessment scenarios. In this paper, we investigate whether structured reasoning strategies can improve the reliability of pretrained MLLMs for post-disaster VQA. Specifically, we explore multiple prompting paradigms in which one MLLM is used to generate task-specific instructions that serve as Chain-of-Thought (CoT) guidance for a second MLLM. These instructions are incorporated during answer generation with varying degrees of in-context learning (ICL), enabling the model to leverage both explicit reasoning guidance and contextual examples. We conduct our evaluation on the FloodNet dataset and compare these approaches against a zero-shot baseline. Our results demonstrate that integrating instruction-driven CoT reasoning consistently improves answer accuracy.

Problem

• post-disaster damage assessment
• multimodal large language models
• visual question answering
• prompt sensitivity
• reliability

Innovation

• Instruction-Guided
• In-Context Learning
• Chain-of-Thought
• Multimodal LLM
• Post-Disaster VQA