DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments

2026-07-19

AI Summary

This study investigates whether intelligent agents can maintain rigorous evidential standards and reason to the correct answer when confronted with realistic, misleading documents that explicitly state incorrect responses. To this end, we introduce a benchmark comprising 100 tasks, each featuring a multi-hop chain of indirect evidence supporting a single correct answer, alongside adversarial distractor documents that directly assert a wrong answer. This benchmark constitutes the first systematic simulation of common misinformation scenarios encountered in open-web environments, exposing a critical flaw in current agents: “verification inertia,” whereby they prematurely accept superficially plausible answers. Experiments reveal that a single misleading document reduces the accuracy of strong baseline agents by 66–88 percentage points. While existing prompting strategies offer partial mitigation, our findings underscore the urgent need for new paradigms that actively reconcile explicit claims with underlying evidential support.

Abstract

Deep research agents increasingly operate over the open web, where relevant records coexist with redundant summaries, outdated reports, and misleading documents. Existing evaluations offer limited insight into whether agents preserve sound evidential standards when an ordinary-looking false document is deliberately seeded into a searchable environment and offers a direct shortcut to a conflicting answer. We introduce DRNOISE, a 100-task benchmark for answer recovery under misleading evidence. Each task has a unique gold answer supported by two corroborating indirect record chains; the paired noisy condition adds one plausible document that states a conflicting answer directly. The benchmark spans ten families of evidence operations. Across agents with strong clean-task performance, this single intervention causes 66-88 percentage-point accuracy drops. Trace analyses identify verification inertia as the dominant failure mode: agents often retrieve truthful records but stop before completing and reconciling the evidence chain, instead deferring to the answer-like document. Generic verification prompts reduce but do not close this gap. The setting is especially relevant to open-web deployment, where plausible falsehoods arrive through ordinary-looking pages rather than explicit attacks. Reliable deep research therefore requires more than retrieval and citation; it requires active reconciliation of direct claims with record-level evidence.

Problem

• misleading evidence
• deep research agents
• evidential standards
• open-web deployment
• answer recovery

Innovation

• DRNOISE
• misleading evidence
• deep research agents
• verification inertia
• evidence reconciliation