Is Deep Research Reliable? Misleading Knowledge Induces False Conclusions
Pengyu Zhu, Lijun Li, Longju Yang, Sen Su, Jing Shao
cs.AI
2026-07-23
Plants fake papers in Deep Research pools. One lifts false-conclusion adoption from 0% to 54.7%; before synthesis, 85.5%. Defenses cut but cannot stop it.
Deep Research is a popular shape for LLM agents: given a question, it decomposes the task, searches repeatedly, accumulates intermediate analysis, and produces a long report. It is being pushed into knowledge-intensive work like scientific research and due diligence. The trouble is that the whole pipeline drinks from the open web, where content is routinely outdated, unsupported, or fabricated. Short-form QA and retrieval-augmented generation have studied what happens when a model is fed misleading context, but Deep Research is a different regime. A planted document can be selected into the workflow, compressed into an intermediate state, and pulled back out several turns later to land in the final report, with nothing checking it along the way. The paper asks whether Deep Research agents can withstand a document that looks authoritative but is fabricated.
The authors built MisKnow-Agent, a framework that manufactures controlled misleading documents to stress-test these agents. For each task in DeepResearch Bench, it fixes a target conclusion that has been manually confirmed false, then generates documents supporting it along two controlled axes: institutional authority (high, medium, low) and style (paper, news, blog, post). Two design choices carry the weight. First, a search-guided refinement loop: every document passes five search-enabled verifier models (GLM-5, Kimi 2.6, DeepSeek-V4 Pro, Qwen3.5-397B, Intern-S1-Pro), and only those all five judge as absent from the real world and therefore false are kept; the rest go back for rewriting until they are unanimously false or the budget runs out. This yields 5,933 quality-controlled misleading documents. Second, the metric FCAR (false-conclusion adoption rate) counts only reports that endorse the false conclusion as their own; mere quotation, summary, hedging, or refutation does not count. The judge is DeepSeek-V4 Pro, validated against humans on 300 reports at 99.7% raw agreement and Cohen's kappa of 0.993.
The systems under test are DeerFlow and WebThinker, each paired with three backbones (DeepSeek-V4 Pro, Qwen3.5-397B, Intern-S1-Pro), plus closed-source Gemini Deep Research. Misleading documents are inserted into isolated retrieval pools, with Serper as the shared search backend.
The headline number is blunt. With no injection, FCAR is 0%; no system produces the task-specific false conclusion on its own. Drop in one misleading document and mean FCAR jumps to 54.7%; two reach 59.5%, three peak at 61.0%, and four or five add nothing. One document is enough; quantity does not compound.
The biggest lever is when the document is seen. Injected at cold start FCAR is 40.5%, during research 44.2%, but immediately before final synthesis it climbs to 85.5%. A fabrication planted at the last step almost always lands in the report.
How it looks matters too. Authority high-to-low moves FCAR from 61.0% to 46.2%, a 14.8-point spread. By style, papers are the most persuasive (61.0%) and posts the least (37.5%), a 23.5-point paper-to-post gap that is larger than the authority gap. Looking like a paper counts for more than the name on it.
| Variable | Setting | Mean FCAR |
| Injection timing | cold / mid / pre-synthesis | 40.5% / 44.2% / 85.5% |
| Authority | high / medium / low | 61.0% / 57.7% / 46.2% |
| Style | paper / blog / news / post | 61.0% / 53.0% / 44.0% / 37.5% |
| Documents | 0 / 1 / 3 / 5 | 0% / 54.7% / 61.0% / 58.3% |
Frameworks differ. Under matched high-authority paper style, DeerFlow averages 67.0% versus WebThinker's 55.0%. The authors trace this to architecture: DeerFlow is a fairly direct planner-researcher-reporter pipeline that passes retrieved documents largely as-is to the reporter, while WebThinker cuts the task into finer subqueries, compresses evidence into document memory, and drafts section by section, which gives it more chances to drop a fabrication mid-flow. The gap shrinks as the backbone's Intelligence Index rises.
Closed-source Gemini Deep Research follows the same direction: 54% for high authority, 54% for paper style, though it is more sensitive to document count (27% at one, 54% at three). The failure mode is not unique to open-source pipelines.
For anyone running or building Deep Research-style products, this reads as a risk briefing. First, five search-enabled models can unanimously flag a document as false, yet the agent still writes its conclusion into the final report; identifying a fake in isolation is not the same as resisting it inside a long workflow. Second, the most dangerous injection point is right before synthesis, which is also where most agents have the thinnest defense. The authors tried two defenses: a pre-research prompt that tells the model to treat retrieved content as unverified and demand independent sources, and a post-research agent that verifies the final report claim by claim. Both cut FCAR (the post-research one down to 20% at its best), but neither reaches zero, and combining them is not always better (on Intern-S1-Pro the combination is worse than either alone). The lesson is direct: verification cannot sit only at the ends of the pipeline. It has to be wired into every evidence-acquisition step and into final synthesis.
A few are visible to the authors and the reader. The task set is only 100 tasks from DeepResearch Bench; only one closed-source system (Gemini) is tested; and the misleading documents sit in an isolated retrieval pool, which is a cleaner simulation than the messier noise of the real open web, where fabrications will not arrive neatly sorted by authority and style. Defense effectiveness varies by model, and the combined defense degrades on Intern-S1-Pro, which the authors attribute to the post-research refiner running in the same contaminated retrieval environment as the original research and possibly re-fetching the fake. Whether these mechanisms hold across a larger task set and more closed-source systems is untested.