Explainable AI Based Diagnosis of Poisoning Attacks in Evolutionary Swarms

2025-05-02

AI Summary

To address collaborative failure in evolutionary swarm systems (e.g., multi-UAV fleets) caused by data poisoning attacks, this paper proposes the first explainable AI (XAI)-driven attack diagnosis framework. Methodologically, it integrates evolutionary game-theoretic modeling with systematic poisoning experiments, introducing a novel strategy-footprint-based attribution mechanism. This mechanism synergistically combines SHAP/LIME interpretability analysis, adversarial poisoning injection, and swarm-level policy representation learning—overcoming the binary (attack/no-attack) limitation of conventional detection methods. Experiments demonstrate 89.7% accuracy in diagnosing strategy anomalies under ≥10% poisoning rates, enabling precise identification of suboptimal cooperation patterns and generating a generalizable set of attack footprint features for real-time online diagnosis. The core contribution is the first application of XAI to poisoning attribution in evolutionary swarms, shifting analysis from “whether attacked” to quantitative, mechanistic interpretation of “how disrupted.”

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Evolutionary Learning

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Models for Web evolution
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Swarming systems, such as for example multi-drone networks, excel at cooperative tasks like monitoring, surveillance, or disaster assistance in critical environments, where autonomous agents make decentralized decisions in order to fulfill team-level objectives in a robust and efficient manner. Unfortunately, team-level coordinated strategies in the wild are vulnerable to data poisoning attacks, resulting in either inaccurate coordination or adversarial behavior among the agents. To address this challenge, we contribute a framework that investigates the effects of such data poisoning attacks, using explainable AI methods. We model the interaction among agents using evolutionary intelligence, where an optimal coalition strategically emerges to perform coordinated tasks. Then, through a rigorous evaluation, the swarm model is systematically poisoned using data manipulation attacks. We showcase the applicability of explainable AI methods to quantify the effects of poisoning on the team strategy and extract footprint characterizations that enable diagnosing. Our findings indicate that when the model is poisoned above 10%, non-optimal strategies resulting in inefficient cooperation can be identified.

Problem

• Diagnosing poisoning attacks in evolutionary swarm systems
• Assessing impact of data manipulation on swarm coordination
• Identifying non-optimal strategies via explainable AI methods

Innovation

• Explainable AI for diagnosing swarm poisoning attacks
• Evolutionary intelligence models agent interactions
• Data manipulation attacks quantify poisoning effects