Retrieval, not hallucinations, will be the limiting factor for LLM-based clinical AI tools
Kirk Roberts, Steven Bedrick, Kurt Miller, William R. Hersh, Hongfang Liu
cs.IR, cs.CL
2026-06-29
A perspective arguing clinical LLM tools are limited by retrieval recall (missed critical history), not hallucination, and recall errors have no equivalent mitigation.
Public discussion of clinical AI is almost entirely about hallucination: the model invents a history that does not exist, gives a diagnosis with no backing. That is a precision error, saying something wrong. This perspective paper, from UTHealth, OHSU, Mayo Clinic, and UT Austin, argues the opposite blind spot: in RAG systems that must first retrieve patient history from the EHR, the more dangerous and more neglected failure is recall error, the omission of critical information that should have been surfaced.
This is a perspective, not an empirical study. The authors borrow the type I (false positive) / type II (false negative) framework from statistics and split errors in clinical LLM tools into two kinds: precision errors are outputs with content the patient data cannot support (hallucination); recall errors are outputs missing information that exists in the patient's full record. They then trace each kind to its source and its mitigation.
Recall error has two sources: the retriever fails to return the critical information (retrieval-level recall failure), or it returns it but the model drops it during generation (generation-level recall failure).
The paper offers no benchmark numbers, only judgments and one small interview study. All 8 clinicians they interviewed raised hallucination as the top concern unprompted; not one had thought about recall error before it was pointed out. After prompting, every one agreed recall error is equally serious.
The core claim is that hallucination can be mitigated by making the model cite its source so the clinician can verify, but recall error has no equivalent remedy. The only backstop is a clinician reading the full record by hand, which defeats the point of using AI to save time. At scale, a system's true recall cannot be measured exactly, only bounded from above, so recall errors may quite literally be a silent killer.
For medical-AI builders this is a redirect: pouring everything into suppressing hallucination may be solving the wrong problem. Retrieval, the question of whether you can surface every critical fact scattered across a very long, structurally complex record, is the harder and more invisible bottleneck, and it will only worsen as records become interoperable and data grows. The paper notes in passing that retrieval has been improving more slowly than LLMs themselves.
The authors concede the type I/II frame does not map cleanly onto generative output, because a model's paragraph-length answer can be wrong in many ways at once. They also admit no large-scale study confirms how much recall error actually hurts, for the very reason that recall is hard to measure. Retrieval benchmarks built on public EHR data (such as MIMIC) may not generalize to an institution's own records, and using an LLM as judge to evaluate retrieval collapses to "retrieval and evaluation are the same problem," an approximation at best.