🤖 AI Summary
To address the observability gap arising from non-deterministic behaviors of LLM-driven AI agents, this paper proposes the first observability framework integrating process mining, causal discovery, and LLM static analysis. Methodologically: (1) process discovery is performed on execution trace logs to model behavioral variation patterns; (2) causal inference identifies root causes of such variations; and (3) static semantic analysis of prompts and code—conducted via LLMs—distinguishes intentional variations (e.g., strategic adjustments) from unintentional ones (e.g., hallucinations or logical drift). Our key contribution is the first synergistic application of process mining and causal discovery for fine-grained, interpretable attribution of AI agent behavior variations. The framework enables developers to locate ambiguous specifications and diagnose unintended execution branches, thereby significantly improving debugging efficiency and system controllability.
📝 Abstract
AI agents that leverage Large Language Models (LLMs) are increasingly becoming core building blocks of modern software systems. A wide range of frameworks is now available to support the specification of such applications. These frameworks enable the definition of agent setups using natural language prompting, which specifies the roles, goals, and tools assigned to the various agents involved. Within such setups, agent behavior is non-deterministic for any given input, highlighting the critical need for robust debugging and observability tools. In this work, we explore the use of process and causal discovery applied to agent execution trajectories as a means of enhancing developer observability. This approach aids in monitoring and understanding the emergent variability in agent behavior. Additionally, we complement this with LLM-based static analysis techniques to distinguish between intended and unintended behavioral variability. We argue that such instrumentation is essential for giving developers greater control over evolving specifications and for identifying aspects of functionality that may require more precise and explicit definitions.