2026-09-17
Paper2Agent turns a paper plus repo into an MCP server. 74 of 100 computational biology papers converted; AlphaGenome tutorial queries hit 98.7% vs 82.7% for Claude on the repo.
A computational paper still ships as a PDF plus a GitHub repo. Using the method on new data means finding the repo, installing dependencies, reading the API, and editing tutorial paths. AlphaGenome makes the tax obvious: clients, variant objects, output modalities. Biologists often spend longer on setup than on the science.
Binder and CodeOcean get an environment to boot. Paper2Code writes code from the paper. Both still ask a human to read and adapt the code. Miao, Pritchard, Zou and colleagues at Stanford turn the paper into a conversational agent that can call its own tools, a stand-in corresponding author.
Paper2Agent compiles a paper into an MCP (Model Context Protocol) server and attaches that server to a chat agent such as Claude Code. MCP is a standard way for an LLM to call external tools and resources. The server has three parts:
The conversion pipeline is itself multi-agent, built on Claude Code's agent SDK. An orchestrator drives five specialists: environment setup, tutorial scanning, tutorial execution, tool extraction, and test-and-repair. It clones the repo, builds an isolated environment, runs tutorials to capture gold outputs, lifts generalizable steps into single-purpose MCP tools, then tests those tools on the tutorial data. A test passes if expected files appear, numbers stay within 3%, and figures match by perceptual hash with Hamming distance under 20. Each function gets at most six repair rounds; repeated failures are dropped from the server so the model cannot improvise broken code that looks scientific. Servers can sit on Hugging Face Spaces, so users skip local installs.
Case studies use Claude Sonnet 4. Several MCP servers can plug into one chat agent, so tools from different papers run in the same session.
AlphaGenome yielded 22 tools, all validated, in about 45 minutes and $14 on a MacBook Air. Baselines were Claude writing code against a local clone (Claude + Repo) and Biomni, Stanford's general biomedical agent. Two experts scored answers against a rubric (inter-rater agreement 96.7%).
| Setting | Paper2Agent | Claude + Repo | Biomni |
| 15 tutorial queries | 98.7±1.3% | 82.7±3.4% | 37.3±4.0% |
| 15 novel queries | 100.0±0.0% | 78.7±4.4% | 56.0±3.4% |
| 30 open-ended research queries | 82.7±2.4% | 56.7±2.3% | 72.2±2.2% |
Median runtime dropped 1.9× and 3.1× versus Claude + Repo and Biomni on tutorial queries, and 2.9× and 3.8× on novel ones. Upgrading Claude + Repo to Claude Opus 4.6 did not close the gap.
Scanpy produced 7 tools in about 45 minutes and $13. On four public single-cell datasets absent from the official tutorials, filtered cell/gene counts and per-cluster markers matched human analyses.
The scale test matters more than the two flagship repos. One hundred computational biology papers were taken in reverse chronological order from bioRxiv's bioinformatics category, with no filter on docs quality. Seventy-four converted end-to-end, proposing 599 tools of which 593 passed tests. Among the 26 failures: missing executable code 34%, missing data or model files 31%, environment or dependency collapse 23%, non-generalizable scripts 12%. A success cost $14.99 and 2.4 hours on average; a failure still cost $9.03 and 1.7 hours, usually dying early. On 300 tutorial questions accuracy was 91.2±1.6%, against 80.3±2.3% for Claude + Repo with Sonnet 4 and 86.3±1.1% with Sonnet 4.6. Queries cost $0.20 and 1.6 minutes, versus $0.38 and 4.3 minutes.
Ten non-biology computational papers (SAM2, TabPFN, SAELens, causal inference, game theory, MCMC) gave 42 execution tasks at 98.1±0.8%. On 26 data-and-discovery papers with no executable tools, the resource layer scored 89.0±3.1% on 100 synthesis questions, against 82.0±3.8% for Claude browsing the paper, at 34× lower cost and 15× lower latency. After shuffling paper-question pairs, out-of-scope rejection was 100%.
Ablations isolate the design. A single agent with a 200k-token window failed to emit a valid MCP in three of three runs. Dropping the test-and-repair agent cut tutorial/novel accuracy to 69.3±4.5% and 84.0±2.7%. Replacing MCP tools with Markdown skill files scored 73.3±2.6% on 30 AlphaGenome questions; MCP tools scored 99.3±0.7%. An OpenCode backend hit 98.7±1.3% tutorial and 97.3±1.6% novel, so the pipeline is not glued to Claude Code.
A discovery case wired three agents together: AlphaGenome, MPRA-coupled single-cell CRISPRi, and CD4+ T cell Perturb-seq, to rank causal genes at the psoriasis locus rs887314. AlphaGenome ranked GPR137 first (RNA-seq quantile score 0.997). A human picked signature correlation from ten agent-proposed strategies. Spearman ρ was 0.613 at Stim8hr and 0.630 at Stim48hr (FDR < 0.05), and not significant at rest (ρ = 0.29, P = 0.21). That cross-screen, cross-modality correlation is not in the source papers.
For method users, this is one step past "the code runs": natural language can call the paper without writing Python. For method authors, repo hygiene now decides whether the paper can be agentified. The authors treat convertibility as a practical reproducibility score, and float an "agent availability" section next to data and code statements.
Paper2Agent is packaging, not a new scientific method. The gain is extraction plus locked tests, not a better model. Tutorial-backed method repos can be converted today.
On open-ended queries Biomni already beats raw-repo Claude (72.2% vs 56.7%). Paper2Agent's 82.7% still leads, but it is no longer near ceiling. Correct tool calls are not correct biology. At the LDL locus the agent favored SORT1; the original paper highlighted CELSR2 and PSRC1. All three have strong liver eQTLs. The paper itself calls causal-gene assignment at such loci hard.
A quarter of the 100 papers still fail. Missing code, missing weights, broken environments: ordinary repo rot. The pipeline treats tutorials as gold; POP-TOOLS still converted after tutorials were stripped, by synthesizing scripts from the README and CLI, but most successes are tutorial-driven.
The psoriasis result is human-in-the-loop. The agent did not choose the strategy, and nobody ran a new wet experiment; existing perturbation screens were recombined. Open-ended scoring against a single reference measures faithful execution, which the paper says is not analytical validity. Security, IP, and credit stay at the level of principles: sandboxed runs, source links, original licenses. MCP servers drift with upstream code. A maintenance agent is proposed, not benchmarked.
A single paper is also the wrong grain for many methods that live across a series. Almost every run sits on Claude; the OpenCode ablation is AlphaGenome-only.