2026-08-24
Stanford's Biomni scores 57% on 443 biomedical questions across 10 tasks and wrote a cloning protocol that succeeded in the wet lab.
Biomedical work is still a pile of disconnected tools. A GWAS means one database, a single-cell run means scanpy, a cloning job means a protocol buried in a methods PDF. Specialist agents exist, but they usually die at the edge of their niche.
Stanford CS researchers around Kexin Huang and Jure Leskovec built Biomni as a generalist: a scientist types a question in plain language, and the system picks tools, writes code, runs the analysis, and returns something a lab can act on. Science published it on 20 August 2026 as Autonomous biomedical research with an artificial intelligence agent. A bioRxiv preprint from June 2025 carries the same architecture. The journal version adds protein-stability design, liquid-handling robots, and a reinforcement-learning recipe. The Science PDF sits behind Cloudflare; methods and the earlier numbers below come from the preprint full text, while the headline scores come from Science's own RESULTS box.
Two pieces. Biomni-E1 is the environment. Biomni-A1 is the agent that lives in it.
To map the action space, an action-discovery agent read 100 recent papers from each of bioRxiv's 25 biomedical subject areas, 2,500 papers in total, and pulled out the tasks, tools, databases, and software needed to reproduce that work. Human experts then kept only the non-trivial items: if a few lines of code would do, it did not become a named tool; every kept tool had to pass a test case. The environment ended up with 150 specialized biomedical tools, 105 software packages, and 59 databases. Databases with a web API (PDB, Open Targets, ClinVar) share a single natural-language query front end; an LLM writes the actual query. Databases without an API were downloaded into a data lake and turned into pandas DataFrames. The runtime speaks Python, R, and Bash.
A1 does not dump the whole catalog into the prompt. A separate LLM retriever first selects a small relevant subset. The agent then writes a numbered plan and executes it with CodeAct: each step is real code, the observation comes back, the plan is revised. Loops, branches, and odd tool combinations go through code rather than a frozen function-calling schema. The Science version also trains Biomni-R0, an open-source model that practices inside E1 with expert-labeled rewards, specifically to climb tool use and multi-step planning.
Science's RESULTS box: 443 questions across 10 biomedical task types, 57% average accuracy, ahead of other agent systems on the same suite. On three expert-level tasks, accuracy matched specialists while taking only a fraction of the time. Detailed recaps of the journal paper fill in the baselines: about 30% for a base LLM and about 44% for a bioinformatics agent given the same tools. Rare-disease diagnosis ran in roughly 3 minutes versus more than 110 for experts; GWAS causal-gene detection in about 4 minutes versus 90.
The preprint splits the older benches. On LAB-Bench database QA, Biomni hit 74.4% against 74.7% for human experts and 40.8% for ReAct+Code. On sequence QA it hit 81.9% against 78.8% human. On a 52-question biomedical slice of Humanity's Last Exam it scored 17.3%, versus 6.0% for the base LLM, 12.8% for a coding agent, and 12.2% for a literature agent. Across eight held-out real-world tasks (variant prioritization, GWAS causal genes, CRISPR screen design, rare-disease diagnosis, drug repurposing, single-cell annotation, microbiome association, patient gene ranking), Biomni beat the base LLM by 402.3% relative, the coding agent by 43.0%, and a ReAct ablation of itself by 20.4%. Typical traces run 6 to 24 steps.
| Setting | Metric | Biomni | Comparator |
| Biomni-Eval1 (Science, 443 items, 10 tasks) | mean accuracy | 57% | base LLM 30%; same-tools coding agent 44% |
| LAB-Bench DbQA (preprint, 315-item subset) | accuracy | 74.4% | human 74.7%; ReAct+Code 40.8% |
| LAB-Bench SeqQA | accuracy | 81.9% | human 78.8% |
| HLE-Bio, 52 questions | accuracy | 17.3% | base LLM 6.0% |
| Biomni-R0-8B after RL | mean task score | 0.59 | 0.32 before RL; Claude 4 Sonnet 0.56 |
| Biomni-R0-32B | mean task score | 0.67 | 8B at 0.59 |
The case studies are closer to a real lab. On 458 wearable files from 30 people, the agent built a 10-step pipeline and reported a 2.19°C mean post-meal temperature rise (median 1.10°C). On 336,000 embryonic skeletal nuclei with paired snRNA and snATAC, a gene-regulatory-network run finished in a little over five hours, recovering the RUNX2 osteogenic axis and flagging AUTS2, ZFHX3, and PBX1 as high-activity regulators across skeletal lineages. Blinded cloning protocols sat near a Stanford genetics postdoc and above a master's trainee. Following Biomni's B2M sgRNA-into-lentiCRISPR v2 Blast protocol produced colonies the next day; both sequenced inserts matched. The journal version adds two more: given only a protein sequence and a request to raise thermostability, the agent composed AlphaFold-2 and ThermoMPNN to propose mutations; given a natural-language wet-lab request, it emitted executable code for a Hamilton STAR liquid handler. After RL, the 8B open model moved from 0.32 to 0.59, above Claude 4 Sonnet's 0.56; the 32B model reached 0.67.
The portable claim is the action space, not a particular frontier model. Swapping different frontier LLMs into the same E1 plus A1 harness still added about 6 to 12 points. The code is open, and a web UI exists. This is useful for labs that already have data and lack someone to wire the tools. It is not a diagnostic oracle.
57% is a fast first-pass analyst with a trace, not an autonomous scientist. A person still has to read the output.
The authors say the evaluated tasks are only a slice of biomedicine; action discovery leaned on recent papers and may miss older methods that still work; database lookup, sequence work, and cloning approach human level, while clinical judgment, novel experimental reasoning, and deep biological synthesis do not; messy multi-omics jobs still want a structured prompt. They also flag biosecurity: a system that writes protocols can be misused.
A few numbers do not sit still. LAB-Bench used a 12.5% subset because of API cost. The preprint says 10 cloning tasks in the results and 20 in the methods. Wearable files are 458 in the results and 485 in the methods. Protein stability, robotics, and R0 details for the journal version rest on the abstract and contemporaneous reports, because the Science PDF was not retrieved. HLE at 17.3% is still low. Open-ended biomedical reasoning is not solved.