DreamGuard: Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model

2026-08-06

AI Summary

This work addresses the limitations of existing runtime protection mechanisms, which are predominantly reactive and struggle to detect long-horizon risks arising from the accumulation of individually benign actions. To overcome this, the paper proposes an active defense system grounded in a risk-aware world model that recursively maintains a compact latent state to forecast future trajectories. By integrating immediate and prefix risk signals across multiple temporal scales, the system enables prospective intervention before action execution. Notably, it introduces an explicit risk evolution modeling mechanism—the first of its kind—that achieves an optimal trade-off between safety and task utility while maintaining low latency (average end-to-end delay of 25 ms). Empirical evaluations demonstrate that the proposed approach significantly outperforms current baselines across four benchmarks and in online assessments.

Abstract

As large language model (LLM) agents increasingly invoke external tools and interact with real-world systems, unsafe actions may cause irreversible consequences on external states, user data, and downstream services. Recent runtime guardrails mitigate such risks by checking proposed actions before execution, but many remain reactive: they primarily assess the apparent safety of the current action, lacking an explicit model of how risk evolves across the trajectory. This limitation creates a critical blind spot for long-horizon risks, where individually benign-looking actions can gradually drift the agent toward hazardous states. In response, we propose DreamGuard, a proactive guardrail for LLM agents built around a risk-aware world model. The world model maintains a compact recurrent latent state over the trajectory and predicts future latent states from which DreamGuard derives immediate-hazard and prefix-risk evidence. It then fuses these multi-horizon signals into intervention decisions before execution. Experiments across four benchmarks and an online guardrail evaluation show that DreamGuard outperforms generic, reactive, and proactive guardrail baselines, achieves the best safety-utility trade-off among evaluated guardrails, and maintains an average end-to-end latency of 25 ms per call.

Problem

• LLM agents
• runtime guardrail
• long-horizon risk
• risk-aware world model
• safety

Innovation

• risk-aware world model
• proactive guardrail
• latent state prediction
• multi-horizon risk assessment
• LLM agent safety