Is Deep Research Reliable? Misleading Knowledge Induces False Conclusions

2026-07-22

AI Summary

This study investigates the vulnerability of deep research agents to drawing erroneous conclusions when exposed to seemingly credible yet factually incorrect information in open-domain settings. To address this, the authors propose MisKnow-Agent, a framework that systematically generates misleading knowledge with controllable authority and stylistic properties, and establishes a multi-level defense evaluation protocol. By integrating large language models, retrieval-augmented generation, evidence synthesis, and verifier models, the framework enables an end-to-end assessment and intervention pipeline. Experiments demonstrate that even minimal exposure to misleading knowledge can induce mainstream agents to adopt false conclusions; while existing verification mechanisms show partial efficacy, they fall short in providing comprehensive protection across the entire reasoning pipeline. Although combined defense strategies mitigate the risk, they cannot fully eliminate it, revealing a pervasive sensitivity of deep research workflows to misinformation.

Abstract

Deep Research agents extend LLM-based assistants into long-horizon workflows involving planning, retrieval, evidence synthesis, and report generation, yet their reliability in open information environments remains underexplored. A key concern is whether apparently credible but factually misleading knowledge encountered in such environments can propagate through these workflows and be adopted as false conclusions in final reports. To study this failure mode, we introduce MisKnow-Agent, a framework for constructing and validating misleading knowledge for Deep Research tasks. MisKnow-Agent generates misleading instances with controllable authority levels and styles, yielding 5,933 quality-controlled instances built on DeepResearch Benchmark tasks. Extensive experiments across open-source and closed-source Deep Research agents show that even limited exposure to misleading knowledge can induce false-conclusion adoption in final reports, revealing a broad reliability vulnerability in current Deep Research agents. Although search-enabled verifier models consistently identify the retained instances as misleading during focused corpus validation, the same instances can still be adopted during long-horizon research, revealing a disconnect between focused verification and workflow-level evidence use. Finally, we evaluate pre- and post-research defenses, both individually and in combination, finding that all three configurations mitigate but do not fully prevent false-conclusion adoption. Our findings suggest that reliable Deep Research requires evidence verification and correction capabilities at both the model and framework levels, beyond improvements in planning, retrieval, evidence integration, or report-generation abilities.

Problem

• Deep Research
• misleading knowledge
• false conclusions
• reliability
• LLM-based agents

Innovation

• Deep Research
• misleading knowledge
• reliability
• evidence verification
• workflow vulnerability