VESTA: A Fully Automated Scenario Generation and Safety Evaluation Framework for LLM Agents

2026-06-07

AI Summary

Existing evaluation methods struggle to comprehensively capture the multidimensional safety risks exhibited by large language model (LLM) agents during task execution. This work proposes the first fully automated, process-level, and multidimensional safety evaluation framework tailored for LLM agents. The framework automatically generates 1,072 measurable scenarios across five core risk dimensions and implements an end-to-end pipeline integrating risk modeling, scenario instantiation, task simulation, and behavioral tracing to enable fine-grained assessment of agent behaviors under varying permission contexts. Evaluation of 12 mainstream LLM agents reveals an average attack success rate of 47.1%, with some models exceeding 70%, underscoring significant behavioral safety vulnerabilities in current systems.

Abstract

Large language models (LLMs) are increasingly evolving from simple text-based interaction systems into LLM agents that can maintain memory, use tools, access external environments, and execute tasks. As their capabilities and autonomy expand, the safety risks they face also become more diverse. Existing evaluations often rely on manually written scenarios, static prompts, or final-output judgments, making it difficult to capture the diverse risks that agents may face during task execution. We introduce VESTA, a fully automated scenario generation and safety evaluation framework for LLM agents. Based on five risk dimensions, VESTA instantiaes abstract and diverse safety risks in real-world task execution into 1,072 measurable evaluation scenarios. Using the automated evaluation pipeline, 12 LLM agents are evaluated under two authority contexts. The results show that current agents still face substantial behavioral safety risks during task execution, with an average ASR of 47.1% and several models exceeding 70%. These findings demonstrate the importance of executable, process-level evaluation for understanding and improving LLM agent safety.

Problem

• LLM agents
• safety evaluation
• scenario generation
• behavioral risks
• task execution

Innovation

• automated scenario generation
• safety evaluation
• LLM agents
• process-level assessment
• behavioral safety risks