Before Reasoning Fails: Pre-Evidence Procedural Failures in Agentic RAG

2026-08-03

AI Summary

This work addresses a critical limitation in current retrieval-augmented generation (RAG) agents, which often generate answers without first verifying retrieved evidence, leading to reasoning failures. The authors decompose such errors into two distinct categories: failures occurring before evidence examination and those persisting even after accessing gold evidence. They frame the omission of evidence usage as a trajectory-level control problem and introduce an evaluation perspective decoupled from answer generation. Through tool-call tracing, entity extraction, Read-Gate runtime constraints, and hidden reasoning budget diagnostics, they demonstrate that these two failure modes are largely non-overlapping. Enforcing mandatory evidence reading significantly improves performance: accuracy on previously skipped trajectories increases by 14.9–19.9 points, and overall minimal reasoning unit accuracy rises by 3.2–9.4 points.

Abstract

Agentic retrieval-augmented generation (RAG) systems can fail before evidence-conditioned reasoning is tested: an agent may retrieve candidate snippets but finalize without inspecting them. We study this failure mode as a procedural property of the agent trajectory, decomposing wrong answers into pre-evidence discipline failures and post-gold-read failures using saved tool-call traces, retrieved evidence, read passages, and final answers. Across 12,000 paired trajectories on HotpotQA, 2WikiMultiHopQA, and MuSiQue, the two failure types are largely non-redundant: the both-trigger rate is in [11.2%, 13.1%] across regex and spaCy entity extractors. We then evaluate Read-Gate, a minimal runtime invariant requiring an agent to read after search and before finalization. Forced reading improves LLM-Acc by 14.9-19.9 points on trajectories that would otherwise skip reading and by 3.2-9.4 points on full minimal-reasoning cells. Additional diagnostics show that larger hidden thinking budgets do not necessarily increase evidence inspection. Together, these results indicate that evidence-gathering should be evaluated as a trajectory-level control problem, separately from answer-side reasoning.

Problem

• Agentic RAG
• pre-evidence failure
• procedural failure
• evidence inspection
• trajectory-level control

Innovation

• Agentic RAG
• pre-evidence failure
• Read-Gate
• trajectory-level control
• evidence inspection