Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)

2026-08-04

AI Summary

This work addresses the lack of robustness evaluation against adaptive red-team attacks and the absence of cybersecurity benchmarks supporting the integration of large language models with reinforcement learning. To this end, we propose Trident, a framework that establishes a dynamic adversarial benchmark and introduces a high-fidelity dataset comprising over 13,000 red-blue interaction trajectories. Trident features a “code-as-policy” agent architecture with three coordinated modules—log summarizer, planner, and encoder—that model red-team attacks as a contextual bandit problem to generate adaptive strategies. By uniquely integrating verifiable reward reinforcement learning (RLVR) with red-teaming, Trident employs a single trainable 7B-parameter planner, reducing blue-team defensive performance by an average of 522% compared to static baselines and autonomously discovering novel adaptive behaviors such as decoy evasion and dynamic state-priority adjustment.

Abstract

Autonomous cyber defense systems based on Deep Reinforcement Learning (DRL) have attracted significant research attention, yet remain evaluated almost exclusively against static, heuristic red agents, leaving their robustness against adaptive threats critically understudied. Meanwhile, recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have improved LLM reasoning, but their integration into cybersecurity remains elusive due to the absence of suitable benchmark environments and interaction datasets. To bridge this gap, we introduce Trident, an agentic LLM red teaming framework comprising three components: a dynamic benchmark with isolated sandbox servers spanning CybORG CAGE 4 and CyberWheel, a dataset comprises over 13,000 high-fidelity red-blue interaction trajectories for RLVR, and a ``Code-as-Policy'' RLVR agentic architecture Trident Agentic). The latter reformulates red agent training as a contextual bandit via a tripartite Log Summarizer--Planner--Coder design, where a trainable Planner generates complete attack strategies from compressed execution logs, which a frozen Coder translates into executable Python policies deployed against live DRL defenders. Empirical evaluations reveal a fundamental brittleness in existing defenses: with a single trainable 7B planner, Trident reduces blue agent defensive performance by an average of 522% compared to static red agent baselines while autonomously discovering emergent behaviors such as decoy avoidance and adaptive state prioritization that static heuristics entirely fail to uncover.

Problem

• Deep Reinforcement Learning
• Autonomous Cyber Defense
• Adaptive Adversaries
• RLVR
• Red Teaming

Innovation

• Trident
• Reinforcement Learning with Verifiable Rewards (RLVR)
• Code-as-Policy
• Agentic Red Teaming
• Dynamic Cyber Defense Evaluation