2026-09-16
Stanford Paper2Agent wraps papers as MCP agents. AlphaGenome hits 15/15 novel queries; 74 of 100 bio papers convert, 91.2% vs 80.3% for Claude on the raw repo.
Methods in computational papers often sit behind a repository. A reader still has to install an environment, learn the API, and match inputs before running the method on new data. Genome foundation models such as AlphaGenome make that barrier concrete: clients, variant objects, and output modalities all have to be wired by hand.
Executable papers, Papers with Code, and Binder lowered the cost of rerunning demos. The knowledge stays static. This Stanford Nature paper turns a paper into a virtual corresponding author. Users ask in natural language; the system calls the paper's own tools, data, and workflows.
Paper2Agent has two layers. Paper2MCP extracts an MCP server from the manuscript and codebase. A chat agent such as Claude Code then sits on top. Each server has three parts:
Specialist agents handle the conversion. An environment agent sets up dependencies. An extraction agent wraps methods as functions. A testing agent checks tutorial data and reference figures, with a 3% numeric tolerance. Failed tools get up to six repair rounds, then they are dropped. Claude Sonnet 4 runs the pipeline. Servers can be hosted on Hugging Face Spaces, and several MCPs can attach to one chat agent.
Baselines are Claude Code writing against the raw repo (Claude + Repo) and the generalist science agent Biomni. Two human experts grade answers with a rubric.
AlphaGenome produced 22 tools in about 45 minutes and $14, all passing automated tests. Accuracy was 98.7±1.3% on 15 tutorial queries and 100.0±0.0% on 15 novel queries, against 82.7%/78.7% for Claude + Repo and 37.3%/56.0% for Biomni. On 30 open-ended researcher queries it scored 82.7±2.4%, versus 56.7±2.3% and 72.2±2.2%. Inter-rater agreement was 96.7%. Median tutorial latency fell by 1.9× versus Claude + Repo and 3.1× versus Biomni.
Scanpy's preprocessing and clustering path produced 7 tools in about 45 minutes and $13. The MCP prompt inspects the data first, then runs the standard pipeline. On public single-cell datasets outside the tutorial repo, cell and gene counts after QC matched human runs, and parameters adapted per dataset.
Scale is the more useful result. Of 100 computational biology papers processed with no manual cleanup, 74 became agents, proposing 599 tools of which 593 passed validation. Failures were mostly missing code, missing artifacts, or broken environments. On 300 tutorial questions, Paper2Agent with Sonnet 4 reached 91.2±1.6%, versus 80.3±2.3% for Claude + Repo on Sonnet 4 and 86.3±1.1% on Sonnet 4.6. Queries cost about $0.20 and 1.6 minutes, versus $0.38 and 4.3 minutes. On 10 non-biology computational papers, 42 execution tasks scored 98.1±0.8%. On 26 discovery-style papers without executable tools, the resource layer scored 89.0±3.1% on 100 synthesis questions, versus 82.0±3.8% for a browser-use Claude, at about 34× lower cost and 15× lower latency. Out-of-scope questions were rejected 100% of the time.
Three agents then looked at the psoriasis locus rs887314. AlphaGenome ranked GPR137 first in CD4+ T cells (RNA-seq quantile 0.997). Matching that prediction to scCRISPRi and Perturb-seq papers, GPR137 knockdown correlated with the CRE perturbation signature at 8h and 48h stimulation (Spearman 0.613 and 0.630, FDR<0.05) but not at rest (0.29, P=0.21). That cross-screen correlation is not a workflow from the source papers.
| Task | Paper2Agent | Strongest baseline |
| AlphaGenome novel queries (15) | 100.0±0.0% | Claude+Repo 78.7% |
| 100-paper tutorial questions | 91.2±1.6% | Claude+Repo 80.3% |
| Non-biology execution tasks | 98.1±0.8% | no tabled baseline |
| Discovery-paper synthesis | 89.0±3.1% | browser 82.0% |
For methods papers with code and tutorials, this is a usable publication form: readers can query, reproduce, and rerun on new data without cloning the repo. Multiple MCPs in one chat let a methods paper and a data paper meet on the fly. The authors also treat "how easily a paper agentifies" as a practical reproducibility score.
This is not a general scientist. Open-ended scores measure faithful execution against a reference, not unique biological truth. In the psoriasis case, the agent proposed strategies; a human picked the correlation analysis.
26 of 100 biology papers still failed, usually from incomplete code, missing data, or dead environments. Adversarial repo-drift repairs some dependency and path bugs, not structurally incomplete repositories. Security, IP, and attribution are flagged and left to the supplement. All applications use Claude, so the lift may not transfer to another coding agent. The GPR137 claim sits on a chosen strategy and a short gene list. The authors keep open-ended scientific judgment human-in-the-loop.