AgentTrails: Towards Trust and Reuse for Agentic Tasks

2026-07-21

AI Summary

This work addresses the challenge that temporal logs of large language model agents executing complex tasks often obscure data dependencies between actions and intermediate artifacts, hindering comprehension, debugging, and reuse. To overcome this limitation, the authors propose a computation provenance–based structured representation that transforms raw execution trajectories into provenance graphs, where tool invocations are modeled as computational actions and their inputs and outputs as data artifacts. By leveraging graph alignment, quotient graph construction, and visualization techniques, the approach enables, for the first time, cross-trajectory data flow alignment, explicit dependency revelation, and automatic extraction of reusable skill patterns. Experimental results demonstrate that the method effectively uncovers hidden dependencies, aligns heterogeneous execution paths, and abstracts generalizable tool usage patterns.

Abstract

LLM-powered agents increasingly tackle complex tasks by invoking tools, querying databases, executing code, and manipulating intermediate artifacts. These agents follow trajectories that are typically stored as chronological logs, obscuring the underlying dataflow -- the dependencies between their actions and the artifacts they create and manipulate. This limits developers' ability to understand the agents' trails, compare executions, debug failures, and re-use the computations. We present AgentTrails, a prototype system for agent provenance and sensemaking. AgentTrails converts raw trajectories into structured provenance graphs, where tool calls are modeled as computational actions and inputs and outputs as data artifacts. The system supports the comparison of executions by placing multiple provenance graphs on a shared canvas and constructing a joined quotient graph that aligns recurring tools, artifacts, and dependency structures across trajectories. On top of this representation, AgentTrails supports pattern extraction, downstream analysis, and skill abstraction. We demonstrate AgentTrails on real-world agent trajectories, showing that it reveals hidden dependencies, aligns divergent executions, and surfaces recurring tool-use patterns beyond chronological logs.

Problem

• agent provenance
• dataflow
• trajectory analysis
• tool-use patterns
• execution comparison

Innovation

• provenance graph
• agent trajectory
• dataflow dependency
• execution comparison
• skill abstraction