Willful Disobedience: Automatically Detecting Failures in Agentic Traces

2026-03-24

AI Summary

This work addresses the limitation of existing evaluation methods that predominantly focus on final outcomes and thus fail to capture procedural violations by AI agents—such as rule-breaking, tool misuse, or deviation from prescribed workflows—during multi-step executions. To this end, we propose AgentPex, the first framework that automatically extracts behavioral specifications from system prompts and validates the compliance of agent execution trajectories against these norms. Integrating natural language rule extraction, behavioral modeling, and trajectory verification, AgentPex enables fine-grained and scalable auditing of agent behavior. Experimental evaluation on 424 real-world trajectories from τ2-bench demonstrates that AgentPex effectively discriminates between models and uncovers numerous process-level violations overlooked by conventional outcome-oriented assessments.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Modeling other Agents
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

AI agents are increasingly embedded in real software systems, where they execute multi-step workflows through multi-turn dialogue, tool invocations, and intermediate decisions. These long execution histories, called agentic traces, make validation difficult. Outcome-only benchmarks can miss critical procedural failures, such as incorrect workflow routing, unsafe tool usage, or violations of prompt-specified rules. This paper presents AgentPex, an AI-powered tool designed to systematically evaluate agentic traces. AgentPex extracts behavioral rules from agent prompts and system instructions, then uses these specifications to automatically evaluate traces for compliance. We evaluate AgentPex on 424 traces from τ2-bench across models in telecom, retail, and airline customer service. Our results show that AgentPex distinguishes agent behavior across models and surfaces specification violations that are not captured by outcome-only scoring. It also provides fine-grained analysis by domain and metric, enabling developers to understand agent strengths and weaknesses at scale.

Problem

• agentic traces
• procedural failures
• behavioral compliance
• AI agent validation
• specification violations

Innovation

• agentic traces
• behavioral rule extraction
• compliance validation
• process-level evaluation
• AgentPex