Multi-Agent Security Tax: Trading Off Security and Collaboration Capabilities in Multi-Agent Systems

2025-02-26

AI Summary

This paper addresses the security risk of multi-hop propagation of malicious commands in multi-agent systems following a single-point compromise, revealing an inherent trade-off between security protection and collaborative performance. Method: We introduce the novel concept of “security tax” to quantify the degradation in collaboration capability induced by security enhancements, and propose a “vaccine-style” defense paradigm based on injectable false memories, comprising two differentiable, general-purpose secure instruction policies. Evaluation is conducted via multi-hop attack modeling and dual-metric assessment (collaboration success rate and attack success rate) in simulation environments. Contribution/Results: Experimental results demonstrate that the proposed vaccine strategies reduce malicious command propagation by 63%, while incurring a 19–34% drop in task collaboration performance—empirically confirming the negative correlation between security and collaboration. This work establishes a theoretical framework and deployable methodology for secure design of multi-agent systems.

Technology Category

• Multiagent Systems: Adversarial Agents
• Intelligent Robots: Multi-Robot Systems
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

As AI agents are increasingly adopted to collaborate on complex objectives, ensuring the security of autonomous multi-agent systems becomes crucial. We develop simulations of agents collaborating on shared objectives to study these security risks and security trade-offs. We focus on scenarios where an attacker compromises one agent, using it to steer the entire system toward misaligned outcomes by corrupting other agents. In this context, we observe infectious malicious prompts - the multi-hop spreading of malicious instructions. To mitigate this risk, we evaluated several strategies: two"vaccination"approaches that insert false memories of safely handling malicious input into the agents' memory stream, and two versions of a generic safety instruction strategy. While these defenses reduce the spread and fulfillment of malicious instructions in our experiments, they tend to decrease collaboration capability in the agent network. Our findings illustrate potential trade-off between security and collaborative efficiency in multi-agent systems, providing insights for designing more secure yet effective AI collaborations.

Problem

• Multi-agent system security risks
• Infectious malicious prompts mitigation
• Security-collaboration trade-offs analysis

Innovation

• False memories vaccination strategy
• Generic safety instruction strategy
• Simulation of multi-agent security risks