From Evidence to Action: How Tool-Using Agents Fail

2026-10-06

AI Summary

Tool-use agents frequently fail by acting on insufficient evidence or when preconditions in multi-step workflows remain unsatisfied. This study elucidates the mechanisms underlying evidence chain fragmentation from decision-making to execution, highlighting fundamental discrepancies between static evaluation and dynamic execution. To address these issues, this work proposes SafeActBench, a novel benchmark that introduces a provenance-bound evidence ledger and a deterministic trajectory evaluator. Through systematic investigation incorporating multi-model configuration testing, workflow dependency tracing, and evidence integrity verification, the results demonstrate that agent failures fundamentally stem from executing actions without establishing sufficient evidence and from inadequately resolving prerequisite dependencies within complex procedural workflows.

Abstract

Tool-using agents make consequential changes to external state, yet correct outcomes do not guarantee that their actions were supported by evidence established beforehand. We study where this evidence-to-action chain breaks as agents move from deciding whether to act to executing single actions and dependent workflows. Across ten model-harness configurations, strong static action assessment can coexist with much weaker interactive execution. Failures often begin before execution: agents stop with incomplete investigation or act before required evidence is established. Once required evidence is obtained, single-action execution is usually reliable, while multi-action workflows additionally expose unresolved prerequisites and incomplete execution. For this analysis, we introduce SafeActBench, comprising 656 cases across six operational domains and five protocols that progress from static action judgment and investigated non-action to single- and multi-action workflows. A provenance-bound Evidence Ledger and deterministic trajectory evaluator track what information was established, when actions occurred, and whether downstream dependencies were satisfied. These results show that failures arise not only from missing information, but also from how agents use established evidence when deciding and executing actions.

Problem

• Tool-Using Agents
• Evidence-to-Action
• Agent Failures
• Multi-Action Workflows
• SafeActBench

Innovation

• Tool-Using Agents
• SafeActBench
• Evidence Ledger
• Trajectory Evaluator
• Multi-Action Workflows