EMBL AI Librarian: Life-Sciences Knowledge Layer for AI Agents
Luigi Sigillo, Matteo Silvestri, Francesco Tabaro, Rajat Bhatnagar, Syed Irtaza Mubashar, Matt Jeffryes, Daljit Nijjer, Vittorio Perera, Ola Spjuth, Julio Saez-Rodriguez, Melissa Harrison, Fabio Petroni
cs.CL, cs.AI, cs.IR, cs.LG
2026-07-30
An LLM-orchestrated layer over Europe PMC gives agents citable evidence in natural language, lifting citation F1 16+ points over baselines and beating web search on LitQA2 by 8.6.
Life-science AI agents all hit the same wall: to reason credibly they have to ground their output in published papers. Summarizing findings across many papers, checking whether a claim holds, or generating testable hypotheses all reduce to finding the right evidence first. The interfaces built to deliver that evidence were designed for humans.
Europe PMC is the open literature search engine for the life sciences: 40.7M PubMed abstracts, 11.9M full-text articles, 1.2M preprints. Its query interface wants keywords and complex syntax, and it hands back whole papers. Biological entities are slippery: the gene TP53 is also p53, and amyotrophic lateral sclerosis goes by ALS plus a string of synonyms and MeSH terms. An agent either learns the query syntax or fires off many complementary searches to catch every spelling, then reads full papers to pull out evidence, and every full text eats into its fixed context window.
The usual fix is a dense vector store: chunk papers into passages, embed them, and retrieve by nearest neighbor. This has real costs. Indexing and serving the whole literature takes hundreds of gigabytes (OpenScholar's store alone is about 744 GB). As models get stronger, dense retrieval's edge over lexical search narrows: a well-tuned BM25 backbone driven by a capable LLM already matches or beats dense retrieval on scientific benchmarks. Embeddings flatten structured metadata, so you cannot filter directly on gene, protein, organism, or chemical fields. And a nearest-neighbor lookup is opaque, unlike a structured fielded query you can read.
EMBL AI Librarian's core choice is to keep no index of its own and query Europe PMC's live search directly, with a single LLM orchestrating the whole pipeline. The system is model-agnostic: any LLM can be the controller, and it inherits gains as models improve. Experiments use a self-hosted GLM-5 (about 700B parameters) on an NVIDIA DGX B200.
Three stages move from recall to precision.
The unit returned is finer than a search result: sentence-level evidence, not whole articles, so downstream agents can quote, cite, and reason over it directly.
Across four complementary benchmarks, agents get better when the Librarian layer is attached.
Literature synthesis (ScholarQABench):
| Setup (synthesis agent = GLM-5) | Citation F1 Bio | Citation F1 Neu |
| OpenScholar-70B (published) | 55.9 | 63.1 |
| PaperQA2 (published) | 56.7 | 56.0 |
| This agent + BM25 (OSDS) | 67.0 | 68.5 |
| This agent + Librarian | 73.8 | 79.4 |
With Librarian, Bio beats the published OpenScholar-70B by 17.9 points and Neu by 16.3, which is the "16+ points" in the abstract. Even within the same agent, swapping only the retriever, Librarian beats the BM25 baseline by 6.8 on Bio and 10.9 on Neu. On the multidisciplinary split, the LLM judge score rises from 4.10 to 4.90, about a fifth.
Claim verification (ProClaim-eval, all agents Claude Sonnet 4.6): switching ProClaim's retriever to Librarian raises agreement with expert consensus from 0.75 to 0.80 (+5). Even a single-prompt verifier agent reaches 0.66 with Librarian, 15 points above retrieving from PubMed and Semantic Scholar. The OpenScholar baseline sits at 0.43.
Open-form QA (LitQA2, same GPT-5.4 backbone): with no retrieval, accuracy collapses to 17.6 from parametric hallucination; web search lifts it to 70.3; Librarian takes it to 78.9 (+8.6), with precision and coverage rising too. Same model, different retriever, so the comparison cleanly isolates the retriever's contribution.
Foundational biology (LAB-Bench): on the stronger GPT-5.4, Librarian mainly lifts coverage, raising macro accuracy from 50.5 to 54.6 (+4.2); on sequence manipulation (SeqQA) accuracy goes 52.5 to 63.8 (+11.3) with coverage jumping 85 to 98.8. On the weaker GPT-4o, Librarian trades coverage for precision: accuracy is essentially flat (35.2 to 35.3), but the model abstains when evidence is thin instead of guessing. Both models stay far below human experts (macro 73.2).
For practitioners, two things carry over. Librarian is a plug-and-play, reusable retrieval layer that helps across all four task types, which suggests evidence-finding can be a shared component every agent reuses rather than rebuilds. And it is concrete evidence for a broader trend: with a strong enough LLM driving it, structured lexical retrieval (BM25 plus fielded queries) can match dense vector retrieval on scientific tasks at a fraction of the infrastructure cost, with no 744 GB index to maintain. Any team weighing whether to build a bespoke vector store for its agents should redo that math.
One honest caveat: some comparisons conflate the retriever with model scale. The synthesis agent uses a roughly 700B-parameter GLM-5 against 70B-class published baselines, and the authors acknowledge this inflates part of the gap. The cleanest retriever-isolated comparisons are ProClaim and LitQA2, where the backbone is fixed and the gains are +5 and +8.6.
The authors list three. Coverage is bounded by Europe PMC, so paywalled full text is missing; they plan to add OpenTargets, UniProt, and ChEMBL. It does not do multi-hop reasoning directly, running a single retrieval round (complementary subqueries help recall but there is no iterative search); multi-hop is left to the downstream agent. And it ignores figures and supplementary material, handling tables only as text.
Two more stand out from reading the paper. Librarian only helps on literature-driven tasks: on LAB-Bench's database-lookup-style questions (DbQA) it adds no decision-relevant evidence, and accuracy ticks down slightly for both models. And the design leans hard on Europe PMC's fielded structure; on a source without structured fields, the "no index, query live" advantage would not hold.