A Systems Engineering Framework for Vision-Language-Enabled UAV Triage and Disaster Response

2026-07-29

AI Summary

This work addresses the limitations of current vision-language models (VLMs) in disaster response, which typically offer only passive information assistance and often increase operator burden rather than facilitating effective human–UAV coordination. To overcome this, the study proposes a novel integrated architecture that embeds a VLM as an active coordination agent within the human–UAV loop, aligning with Incident Command System (ICS) protocols and enabling natural language interaction and task-level coordination. The system design is guided by Model-Based Systems Engineering (MBSE) principles and validated through software-in-the-loop simulation and human factors evaluation. Preliminary experiments demonstrate that the proposed framework significantly reduces operators’ mental workload, effort, and frustration while enhancing trust in AI and communication clarity, thereby achieving a paradigm shift from passive information support to proactive coordination.

Abstract

Recent advances in Vision Language Models (VLMs) have created new opportunities for disaster response, where responders must interpret large volumes of sensor data under time pressure. Current VLM applications include social media monitoring for situational awareness, generation of draft action plans, and translation of technical alerts into public-facing messages. While these efforts can accelerate information flow, they remain largely limited to decision-support roles. Such approaches can increase operator burden because humans must still translate outputs into coordinated actions across teams and robotic assets. This study explores the viability of embedding VLMs as coordination agents within the human-UAV loop. The proposed architecture integrates natural language interaction, mission-level task coordination, software-in-the-loop implementation, and communication aligned with the Incident Command System (ICS). Rather than functioning solely as advisory tools, VLMs facilitate communication between human operators, mission control logic, and UAV task execution. The framework was developed using a Model-Based Systems Engineering (MBSE) approach, with use case and block definition diagrams representing system roles, internal structure, and component interactions. Three key elements, the VLM Coordinator Agent, UAV Mission Control, and Task Allocator, were implemented within an integrated simulation and control environment. A preliminary human-factors evaluation with seven participants showed reduced perceived workload across mental demand, effort, and frustration, along with high ratings for AI trust and communication clarity. By integrating MBSE, software-in-the-loop testing, and human-factors evaluation, this work advances scalable human-autonomy teaming for high-stakes disaster response, with broader implications for aerospace autonomy and civil safety.

Problem

• Vision-Language Models
• Disaster Response
• Human-Autonomy Teaming
• UAV Coordination
• Operator Workload

Innovation

• Vision-Language Models
• Model-Based Systems Engineering
• Human-Autonomy Teaming
• UAV Coordination
• Incident Command System