GPT-Red: Automated Red Teaming via Self-Play at Scale

2026-07-28

AI Summary

This work addresses the vulnerability of large language models to prompt injection attacks by introducing GPT-Red, an automated red-teaming agent based on large-scale self-play reinforcement learning. The approach employs multi-agent adversarial training to co-evolve attack strategies against a diverse population of defenders, achieving a scale of security training comparable to the largest reinforcement learning post-training efforts to date and establishing a flywheel mechanism for mutual improvement of attacks and defenses. Experiments demonstrate that GPT-Red reliably compromises GPT-5.5 and earlier models, surpassing human red teams in attack success rate, and exhibits strong generalization across unseen environments, defense models, and interfaces. The framework has already been successfully integrated into the adversarial training pipeline for GPT-5.6.

Abstract

We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluate and improve the robustness of our production systems. To this end, we use it to adversarially train GPT-5.6, our most robust model to prompt injections to date. To create GPT-Red, we design a scalable self-play algorithm where the model is tasked with attacking a diverse population of simultaneously-trained defender agents. We train the model on realistic red-teaming environments using compute on the same scale as some of our largest RL post-training runs, making it the single-largest LLM safety training run ever documented. GPT-Red excels at red-teaming: it reliably breaks our past models up to GPT-5.5, it finds more successful attacks than human red-teamers, and it generalizes to held-out environments, defender models, and harnesses. In the future, we expect that as we improve the robustness of each new GPT model, it will in turn will provide better learning signal for \textit{even stronger} red-teamer agents, thus unlocking a self-improvement flywheel.

Problem

• red teaming
• prompt injection
• large language models
• adversarial robustness
• automated attack discovery

Innovation

• automated red teaming
• self-play
• prompt injection
• adversarial training
• LLM safety