🤖 AI Summary
Current AI agents lack effective security evaluation mechanisms against indirect prompt injection attacks, particularly missing a reconnaissance phase analogous to that in traditional penetration testing. This work formally defines the reconnaissance process for AI agents and introduces KYA, a reconnaissance-driven, black-box penetration testing framework. KYA actively probes target agents to construct detailed profiles, identifies and extracts their knowledge assets, and leverages this intelligence to generate more potent, tailored attacks. The end-to-end framework demonstrates strong effectiveness across multiple agent security benchmarks and real-world coding agents. To facilitate further research, the authors open-source the KYA framework, along with a dedicated benchmark dataset and baseline implementations.
📝 Abstract
Traditional pentesting uses reconnaissance at each step to uncover unseen weaknesses, build stronger attacks, and advance the objective; we argue that AI agents require the same treatment. We formalize agent reconnaissance by modeling the process and identifying the knowledge assets it seeks to extract: what they are, how they are used, and which agent weaknesses they exploit to give adversaries leverage in indirect prompt injection attacks. We instantiate these insights in Know Your Agent (KYA), a framework that automates black-box, reconnaissance-driven pentesting by probing agents, building target profiles, and using those profiles to craft stronger attacks. We evaluate KYA on agent-security benchmarks and a real-world coding agent, and release KYA, its benchmarks, and baseline implementations for reproducibility.