Factorized Deep Q-Network for Cooperative Multi-Agent Reinforcement Learning in Victim Tagging

2025-03-02

AI Summary

In mass-casualty incidents (MCIs), distributed triage by multi-agent systems faces challenges including prolonged triage time, constrained communication bandwidth, and environmental uncertainty. Method: This paper formulates a decentralized decision-making problem aimed at minimizing the total triage completion time under realistic operational constraints. We propose Factorized Decoupled Deep Q-Networks (FDQN), the first application of FDQN to emergency-response multi-agent coordination under real-world constraints, and design a distributed heuristic baseline supporting local communication and dynamic replanning. Contribution/Results: Multi-scale simulations show FDQN reduces average triage time by 18.7% in small-scale scenarios, while the heuristic demonstrates superior robustness in complex, large-scale settings. The study delineates the practical applicability boundary of multi-agent reinforcement learning (MARL) in emergency response, establishing a novel architecture and evaluation paradigm for deployable intelligent emergency decision-making systems.

Technology Category

• Multiagent Systems: Multiagent Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Decentralized Web and Fediverse systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Mass casualty incidents (MCIs) are a growing concern, characterized by complexity and uncertainty that demand adaptive decision-making strategies. The victim tagging step in the emergency medical response must be completed quickly and is crucial for providing information to guide subsequent time-constrained response actions. In this paper, we present a mathematical formulation of multi-agent victim tagging to minimize the time it takes for responders to tag all victims. Five distributed heuristics are formulated and evaluated with simulation experiments. The heuristics considered are on-the go, practical solutions that represent varying levels of situational uncertainty in the form of global or local communication capabilities, showcasing practical constraints. We further investigate the performance of a multi-agent reinforcement learning (MARL) strategy, factorized deep Q-network (FDQN), to minimize victim tagging time as compared to baseline heuristics. Extensive simulations demonstrate that between the heuristics, methods with local communication are more efficient for adaptive victim tagging, specifically choosing the nearest victim with the option to replan. Analyzing all experiments, we find that our FDQN approach outperforms heuristics in smaller-scale scenarios, while heuristics excel in more complex scenarios. Our experiments contain diverse complexities that explore the upper limits of MARL capabilities for real-world applications and reveal key insights.

Problem

• Minimize victim tagging time in mass casualty incidents.
• Evaluate heuristics for adaptive decision-making under uncertainty.
• Compare FDQN performance with baseline heuristics in MARL.

Innovation

• Factorized Deep Q-Network for MARL
• Local communication enhances efficiency
• FDQN outperforms heuristics in small-scale