AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations

2026-06-01

AI Summary

This work addresses indirect prompt injection attacks against large language model (LLM) agents induced by third-party service responses in SaaS tool integrations. To counter this threat, we propose AGENTREDGUARD, a novel defense framework featuring an integration-aware mitigation mechanism and a version-locked evaluation protocol. We also introduce the first dynamic red-teaming benchmark specifically designed for tool responses, encompassing 24 enterprise-grade integrations, 9 functional categories, 5 attack types, and 215 fine-grained authorization boundary scenarios. Experimental results demonstrate that our approach reduces the average attack success rate across eight mainstream LLMs from 69.9% to 2.4%, with a false positive rate of only 0.37%, substantially outperforming existing open-source defenses.

Abstract

Indirect prompt injection in tool-use agents is a concrete production threat: LLM agents read from integrations (third-party services such as Gmail, Salesforce, or Jira accessed through tool calls) whose response content the user neither writes nor controls. Existing benchmarks under-measure the threat: most cover only a handful of integrations with the same attack payload replayed across runs, and open-source guards are trained on chat-style data rather than tool-response content. We introduce AGENTREDBENCH, a dynamic LLM-driven redteaming benchmark of 215 subtle underspecified authorization (attacks at the boundary of what the user's request authorises) scenarios across 24 enterprise integrations in nine functional families and five attack types. Across an eight-model panel (Anthropic, OpenAI, Google), no-guard ASR (attack success rate) ranges from 32% (Claude Sonnet 4.6) to 81% (Gemini 3 Flash). To keep the scenario set out of training corpora and preserve headline ASR meaning over time, we release the codebase, integration schemas, and AGENTREDGUARD model openly; the canonical scenarios are evaluated through a maintainer-mediated channel with immutable versioning. We release AGENTREDGUARD alongside the benchmark: a guard trained on an integration-diverse corpus of adversarial tool-response content. AGENTREDGUARD cuts panel ASR from 69.9% to 2.4% at 0.37% false-positive rate, outperforming every open-source baseline with non-trivial detection (Llama Guard, PromptGuard 2, ProtectAI) on both axes. Cross-integration and cross-attack type holdouts both confirm the gain transfers beyond the training subset.

Problem

• indirect prompt injection
• LLM agents
• SaaS integrations
• authorization boundary
• redteaming

Innovation

• indirect prompt injection
• tool-use agents
• dynamic redteaming
• integration-aware defense
• adversarial tool-response