ATAG: AI-Agent Application Threat Assessment with Attack Graphs

2025-06-03

AI Summary

Existing attack graph models fail to capture composite attack paths—such as prompt injection, excessive autonomy, and sensitive information leakage—in LLM-driven multi-agent systems (MAS), hindering rigorous security assessment. Method: We propose ATAG, a novel framework featuring (i) a logic-based attack graph generation method tailored to agent topology and dynamic interactions; (ii) the first standardized LLM Vulnerability Database (LVD); and (iii) an extended MulVAL engine integrating domain-specific facts and agent interaction rules, unifying formal modeling with empirical analysis. Contribution/Results: Evaluated on two real-world MAS deployments, ATAG automatically infers multi-step, cross-agent attack chains, enabling precise identification and prioritization of high-risk paths. It significantly enhances the interpretability and practicality of threat analysis for LLM-MAS, establishing a foundation for systematic, scalable security evaluation.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Evaluating the security of multi-agent systems (MASs) powered by large language models (LLMs) is challenging, primarily because of the systems' complex internal dynamics and the evolving nature of LLM vulnerabilities. Traditional attack graph (AG) methods often lack the specific capabilities to model attacks on LLMs. This paper introduces AI-agent application Threat assessment with Attack Graphs (ATAG), a novel framework designed to systematically analyze the security risks associated with AI-agent applications. ATAG extends the MulVAL logic-based AG generation tool with custom facts and interaction rules to accurately represent AI-agent topologies, vulnerabilities, and attack scenarios. As part of this research, we also created the LLM vulnerability database (LVD) to initiate the process of standardizing LLM vulnerabilities documentation. To demonstrate ATAG's efficacy, we applied it to two multi-agent applications. Our case studies demonstrated the framework's ability to model and generate AGs for sophisticated, multi-step attack scenarios exploiting vulnerabilities such as prompt injection, excessive agency, sensitive information disclosure, and insecure output handling across interconnected agents. ATAG is an important step toward a robust methodology and toolset to help understand, visualize, and prioritize complex attack paths in multi-agent AI systems (MAASs). It facilitates proactive identification and mitigation of AI-agent threats in multi-agent applications.

Problem

• Assessing security risks in multi-agent systems with LLMs
• Extending attack graphs to model AI-agent vulnerabilities
• Standardizing documentation for LLM vulnerabilities and threats

Innovation

• Extends MulVAL with custom facts for AI-agent topologies
• Creates LLM vulnerability database (LVD) for standardization
• Models multi-step attacks like prompt injection across agents