Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

2026-07-20

AI Summary

This work addresses the limitation of current large language model (LLM) safety evaluations, which predominantly rely on static, single-turn attacks and fail to capture real-world dynamic threats. The authors propose the first benchmark supporting multi-turn adaptive attacks, encompassing 21 scenarios to assess memoryless LLM defenders against autonomous LLM attackers that observe prior responses and dynamically adjust their strategies over up to 15 interaction rounds. Key innovations include a multi-LLM collaborative attack framework, structured scoring, cosine similarity analysis, and a replayable attack dataset. Experiments reveal that attack success rates rise from near 0% in single-turn settings to 5.4%–14.0% after 15 rounds, with ensembles of three attackers uncovering 1.4–2.2 times more successful attacks than the best individual attacker. Notably, Claude Opus 4.6 and GPT-5.4 exhibit comparable overall performance but distinct vulnerability patterns.

Abstract

LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. Most safety benchmarks evaluate defenders against fixed attack pools collected before evaluation, single-turn or multi-turn. We present a 21-scenario benchmark for \emph{adaptive multi-round attacks against memoryless LLM defenders}: an autonomous LLM attacker observes prior defender responses and pivots across rounds, while each defender response is evaluated as a fresh interaction. Holding the 21 scenarios, attackers, defenders, and structured-output scoring fixed, restricting scoring to the first attacker turn yields $0$-$1\%$ attack success rate (ASR); allowing 15 rounds of adaptive attack yields $5.4$-$14.0\%$. Pooling three frontier attacker LLMs uncovers $1.4$-$2.2\times$ as many unique successful attacks as the best single attacker, and the generated attacks have low cosine similarity ($0.02$-$0.14$) to attacks in existing benchmarks. Claude Opus 4.6 and GPT-5.4 are tied in aggregate ($5.4\%$ each; overlapping $95\%$ CIs), but their weaknesses differ sharply: on one scenario Opus reaches $60\%$ ASR ($95\%$ CI $36$--$80\%$) while GPT-5.4 and Gemini each stay at $7\%$ (CI $1$-$30\%$; the gap is preserved in a higher-$N$ replication). $13$ of $21$ scenarios distinguish at least one defender pair, yet rankings disagree across scenarios (Kendall's $W = 0.19$). We release the benchmark -- 21 evaluation scenarios, 10 public development scenarios, the orchestrator, baseline harnesses, and a multi-attacker CLI -- plus 945 transcripts from the 3$\times$3 frontier matrix, an attack-replay dataset, and 18{,}422 gpt-oss-20b battles from an open competition's final scoring rounds.

Problem

• adaptive attacks
• multi-turn manipulation
• LLM agent security
• prompt injection
• memoryless defenders

Innovation

• adaptive adversarial attacks
• multi-turn LLM agents
• LLM security benchmark
• memoryless defenders
• structured-output evaluation