From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents

2026-06-04

AI Summary

This work addresses the limitations of existing safety mechanisms for large language model agents, which typically apply coarse-grained interception across entire tasks and lack closed-loop validation of downstream behaviors. To overcome these challenges, the authors propose TRIAD, a novel framework featuring a ternary decision mechanism—Execute, Reject, Update—that integrates structured natural language feedback to dynamically refine agent action plans during the planning phase. This approach enables fine-grained risk intervention while preserving task objectives. By fine-tuning language models on a custom dataset and embedding safety feedback into the agent’s context to support iterative plan revision, TRIAD reduces the average attack success rate to 10.42% on the ASB and AgentHarm benchmarks, achieving the current best trade-off between safety and task utility.

Abstract

LLM-based guardrails typically safeguard agents by evaluating proposed actions or inputs before execution, producing safety signals such as binary allow/deny decisions, risk categories, and/or explanatory rationales about potential policy violations. However, agent risks often arise when otherwise benign tasks are contaminated by untrusted external content, unsafe instructions, or risky tool use. Existing guardrails often flag the entire task uniformly as unsafe, thereby blocking the threat but sacrificing the benign part. Moreover, existing work largely evaluates guardrails in isolation, leaving unclear whether their interventions lead to safer downstream agent behavior. To address this, we introduce TRIAD (Tripartite Response for Iterative Agent Guardrailing), a guardrail-integrated agent framework that leverages guardrail-generated verbal feedback as a guiding signal to keep the agent aligned with benign objectives at each planning step. We finetune a language model on a self-curated training dataset to output one of three decisions: proceed, refuse, or update, together with structured natural-language feedback. Rather than merely allowing or blocking execution, update guides the agent to revise its plan, avoid harmful components, and preserve the benign task where possible. TRIAD injects this feedback into the agent's context, enabling subsequent plan revision and forming a closed loop between guardrail feedback and agent planning. Extensive experiments on ASB and AgentHarm show that TRIAD reduces the average attack success rate to 10.42%, while achieving the best safety-utility trade-off among guardrail-integrated baselines. Our code is available at: https://github.com/YUHAOSUNABC/TRIAD.

Problem

• LLM agents
• guardrails
• risk classification
• action plan remediation
• safety-utility trade-off

Innovation

• guardrail feedback
• LLM agents
• iterative planning
• safety-utility trade-off
• tripartite decision