What Did the Agent Actually Do? Evidence-Grounded Oversight for Long-Horizon Agents

2026-10-05

AI Summary

This study addresses the challenge of fragmented evidence during long-horizon agent execution, which hinders users from identifying critical decisions. To this end, it introduces AgentMonBench, a dedicated evaluation benchmark, and proposes the first training-free Evidence-linked Behavior Graph (EBG). Employing unsupervised graph-structured modeling, EBG clusters source-linked evidence into behavior nodes to construct relational graphs, thereby enabling task-oriented behavioral context interpretation and automated localization of critical decisions. Experimental results demonstrate that EBG significantly improves the accuracy of decision identification and evidence localization across eight foundation models, while maintaining stable performance as input scale increases.

Abstract

As agents take on long-horizon tasks, users shift from making individual decisions to overseeing autonomous execution. Yet the volume of agent activity and the fragmentation of supporting evidence make it difficult to determine which decisions warrant user verification. We study monitors that identify consequential decisions and locate evidence to help users assess their implications. We introduce AgentMonBench, a software-engineering benchmark comprising three subsets that cover two complementary dimensions: alignment between requirements and behavior, and awareness of consequential autonomous decisions for verification. To support these judgments, we propose the Evidence-Grounded Behavior Graph (EBG), a training-free method that groups source-linked evidence into behaviors and organizes their relationships into a graph. EBG presents task-oriented views of this graph to help monitors interpret behavior in context. Experiments across eight models show that EBG improves decision identification and evidence localization in most settings compared with direct access to the original context. Further experiments show that EBG's evidence-localization gains persist across input scales and hyperparameter settings, while real-world applications illustrate its practical value for human oversight.

Problem

• long-horizon agents
• human oversight
• evidence localization
• consequential decisions
• autonomous execution

Innovation

• Evidence-Grounded Behavior Graph
• AgentMonBench
• Long-Horizon Agents
• Decision Identification
• Evidence Localization