2026-09-17
Paper2Agent turns a paper plus its repo into a locked MCP tool server. On AlphaGenome, novel queries hit 100%; 74 of 100 computational-biology papers agentified.
Computational papers still dump most of the work on the reader. Finding the repo, installing dependencies, learning the API tree, then wiring the method onto new data is the actual product. AlphaGenome is the extreme case: client objects, variant records, output modalities. A biologist who could use the model often never gets past setup.
Executable papers, Papers with Code, Binder, and CodeOcean already ship environments. Users still have to read the code and sequence the analysis themselves. Paper2Agent treats the paper as a virtual corresponding author. Manuscript, supplements, datasets, code, and workflows become tools an LLM agent can call.
A Stanford team (Miao, Pritchard, Zou and colleagues) built a multi-agent pipeline on Claude Code's agent SDK. The artifact is a remote MCP server. MCP, the Model Context Protocol, is a standard way for LLMs to call tools and data.
Each server has three parts:
Six stages follow. Locate and clone the repo; an environment agent builds an isolated setup; a scanner indexes tutorials; an executor runs them as gold-standard traces; an extractor turns steps into parameterized functions and a verifier tests them; the orchestrator packs survivors into an MCP and attaches a chat agent (Claude Sonnet 4 in the experiments). A test passes if expected files appear, numbers stay within 3%, and figures match by perceptual hash with Hamming distance under 20. Each function gets at most six repair rounds. Repeated failures are stripped from the server so the chat model is not asked to improvise science code.
AlphaGenome is the flagship. On a laptop, about 45 minutes and $14 produced 22 tools, all of which passed. Baselines were Claude writing code against the raw repo (Claude + Repo) and Stanford's generalist scientist agent Biomni. Fifteen tutorial queries and fifteen novel ones, five runs each, graded by two experts (96.7% agreement).
| Method | Tutorial | Novel | 30 open-ended |
| Paper2Agent | 98.7±1.3% | 100.0±0.0% | 82.7±2.4% |
| Claude + Repo | 82.7±3.4% | 78.7±4.4% | 56.7±2.3% |
| Biomni | 37.3±4.0% | 56.0±3.4% | 72.2±2.2% |
Median latency dropped 1.9× versus Claude + Repo and 3.1× versus Biomni on tutorial queries, 2.9× and 3.8× on novel ones. The gap held under prompt paraphrase and when the repo baseline moved to Claude Opus 4.6.
Scanpy's preprocessing and clustering path produced seven tools in about 45 minutes and $13. On four public single-cell datasets, cell counts, gene counts, and cluster markers matched a human-run pipeline. The TISSUE spatial-transcriptomics paper was wrapped the same way and matched the human tutorial, with details in the extended data.
Scale is the more honest test. Of 100 computational biology papers sampled backward through bioRxiv's bioinformatics category with no quality filter, 74 became agents: 599 tools proposed, 593 validated. Failures clustered on missing code, missing data or weights, broken environments, and scripts that do not generalize. On 300 tutorial questions Paper2Agent plus Sonnet 4 scored 91.2±1.6%, against Claude + Repo at 80.3±2.3% with Sonnet 4 and 86.3±1.1% with Sonnet 4.6. Queries cost about $0.20 and 1.6 minutes, versus $0.38 and 4.3 minutes. Ten non-biology computational papers (SAM2, TabPFN, causal forests, MCMC, and others) reached 98.1±0.8% on 42 execution tasks. On 26 data-heavy papers that could not yield tools, the resource layer scored 89.0±3.1% on 100 synthesis questions, beating Claude browsing the paper (82.0±3.8%) at 34× lower cost and 15× lower latency. A permuted paper-question test was rejected 100% of the time.
A discovery demo wired AlphaGenome, an MPRA-scCRISPRi screen, and a CD4+ T-cell Perturb-seq paper to rank causal genes at the psoriasis SNP rs887314. AlphaGenome put GPR137 first (RNA-seq quantile 0.997). After a human chose signature correlation, only GPR137 knockdown tracked the CRE perturbation under stimulation (Spearman 0.613 and 0.630, P = 3.79×10⁻³ and 4.71×10⁻³, FDR < 0.05), not at rest (ρ = 0.29, P = 0.21).
For consumers of methods, this is closer to talking to the corresponding author than to cloning a repo. Tools are frozen after validation, which cuts a major source of code hallucination. For producers, whether a paper wraps cleanly is a practical reproducibility score. Because MCPs compose, a method agent and a data agent can run analyses the source papers never wrote. The psoriasis signature-correlation step is one of those.
This is still systems work. The substrate is Claude Code. The contribution is orchestration, locked tests, and a standard envelope for paper assets, not a new reasoning algorithm.
26 of 100 papers never wrapped. Missing code, missing artifacts, and dead environments remain blockers. Open-ended scientific reasoning stays human-in-the-loop for choosing directions and weighing evidence. The authors say agreement with a single reference on open-ended tasks measures faithful execution, not analytic correctness. On the LDL GWAS example the agent ranked SORT1 above CELSR2 and PSRC1, which the original AlphaGenome paper highlighted; all three have extreme quantile scores and strong liver eQTLs, so the new reading is not a settled causal call. Almost all numbers use Sonnet 4; OpenCode and Markdown-skill ablations live in the supplement. Executable agents also create security, IP, and attribution problems, and they rot when upstream repos move.