2026-09-24
Paper2Agent wraps papers as MCP agents. AlphaGenome: 22 tools, 45 min, $14, 98.7% on tutorials. 74 of 100 computational biology papers converted; 91.2% on 300 questions.
Once a computational paper is out, a reader still has to find the repo, install it, learn the API, and retarget the examples. AlphaGenome makes the cost concrete: clients, variant objects, output modalities, tissue ontologies. A well-kept repository does not remove that engineering tax for biologists.
Binder, CodeOcean, Papers with Code, and executable papers made code runnable. Paper2Agent, from Stanford's Jiacheng Miao, James Zou and colleagues, wraps the manuscript, code, data and workflows as a conversational agent that acts as a virtual corresponding author. You ask in natural language; the agent calls tools that were extracted from the paper and locked against its own outputs.
The carrier is MCP (Model Context Protocol), a standard for letting an LLM call external tools. Each paper becomes one MCP server with three parts:
The system is a multi-agent pipeline on Claude Code's agent SDK, with Claude Sonnet 4 as the base model throughout. An orchestrator drives six steps: locate and clone the repo, build an isolated environment, find tutorials, run them end to end as gold standard, extract reusable functions and test them against tutorial outputs, then assemble the MCP server.
A test passes if expected files appear, numeric results sit within 3% of the tutorial, and figures match by perceptual hash (Hamming distance under 20). Each function gets at most six repair rounds; failures are stripped from the final server. Servers can live on Hugging Face Spaces and plug into any compatible chat agent.
The lock is there to stop code hallucination. Tools are frozen against the original repository's results rather than generated on the fly.
On AlphaGenome, Paper2Agent produced 22 MCP tools, all of which passed validation, in about 45 minutes for $14 on a laptop. Over five runs it scored 98.7±1.3% on 15 tutorial queries and 100.0±0.0% on 15 novel ones. Claude Code writing against the raw repo scored 82.7±3.4% and 78.7±4.4%; Biomni scored 37.3±4.0% and 56.0±3.4%. On 30 open-ended research queries that need tool composition and a biological conclusion, Paper2Agent reached 82.7±2.4%, versus 56.7±2.3% for Claude+Repo and 72.2±2.2% for Biomni. Median latency on tutorial queries was 1.9× faster than Claude+Repo and 3.1× faster than Biomni. Two experts graded with a preset rubric; inter-rater agreement was 96.7%.
| Setting | Paper2Agent | Claude+Repo | Biomni |
| AlphaGenome tutorials (15) | 98.7±1.3% | 82.7±3.4% | 37.3±4.0% |
| AlphaGenome novel (15) | 100.0±0.0% | 78.7±4.4% | 56.0±3.4% |
| Open-ended research (30) | 82.7±2.4% | 56.7±2.3% | 72.2±2.2% |
| 300 tutorial Qs, 100 bio papers | 91.2±1.6% | 80.3±2.3% (86.3% on Sonnet 4.6) | not run |
Scanpy's preprocessing and clustering path produced seven tools in about 45 minutes for $13. On four public single-cell datasets that are not in the Scanpy repo, cell counts, gene counts and cluster markers matched a human following the official tutorial.
Scaled to 100 computational biology papers sampled backward from bioRxiv's bioinformatics category with no quality filter, 74 finished extraction, execution and automated validation. Of 599 proposed tools, 593 passed. Per query it cost $0.20 and 1.6 minutes, versus $0.38 and 4.3 minutes. Ten non-biology computational papers (causal inference, computer vision, game theory, astrophysical MCMC and others) scored 98.1±0.8% on 42 execution tasks. On 26 data-and-discovery papers the resource layer hit 89.0±3.1% on 100 synthesis questions, 34× cheaper and 15× faster than Claude browsing the paper page. A shuffled paper-question out-of-scope test was rejected 100% of the time.
Three agents were chained on the psoriasis variant rs887314. AlphaGenome ranked GPR137 first in CD4+ T cells (RNA-seq quantile 0.997). Cross-checked against MPRA-scCRISPRi and Perturb-seq, GPR137 knockdown signatures correlated with the CRE perturbation signature under stimulation (Spearman 0.613 at Stim8hr, P=3.79×10⁻³; 0.630 at Stim48hr, P=4.71×10⁻³; FDR<0.05). BAD and three other candidates did not. A human picked signature correlation from ten strategies the co-scientist proposed.
For anyone trying to reuse a method, the README-and-install loop becomes a question. Locking tools to the original repo is more reliable than letting a general agent write code against a clone. MCP is modular, so several paper agents can sit in one chat session.
This is an engineering wrapper, not a new model. It needs public code and tutorials; it fails when the repo itself is not reproducible. Treating "can this paper be agentified" as a reproducibility metric is a fair claim. Twenty-six of 100 papers still failed, so it is not a one-click converter for arbitrary papers.
The authors list the failure modes: incomplete code, missing data or model files, broken environments, scripts that do not generalize. Open-ended reasoning stays human-in-the-loop. Open questions can have several defensible answers; agreement with a single reference mainly measures faithful execution, not scientific validity.
The headline model is Sonnet 4, shared with the baselines, so the gain is the MCP wrap. They did show Claude+Repo still trails after upgrading to Opus 4.6 and Sonnet 4.6. The main tables do not switch off Anthropic scaffolding (an OpenCode ablation sits in the supplement). On the LDL variant chr1:109274968:G>T the agent prefers SORT1 while the original AlphaGenome paper highlighted CELSR2 and PSRC1; all three have quantile scores above 0.999 and significant liver eQTLs. The authors treat that as the usual GWAS ambiguity, not as the agent correcting the paper. Security, IP, attribution, and who maintains an agent when the upstream repo moves, are named and left open.