BixBench3: A benchmark for AI agents to reproduce biology paper analyses
ZhongingAlong · x · 2026-08-27
Edison Sciences released BixBench3, a long-horizon biology benchmark designed to evaluate an LLM's ability to produce the analyses underpinning entire biology papers from raw data.
- Scope: It features 20 paper-derived tasks with datasets up to 241 GB, making it the longest-horizon biology benchmark in terms of tokens, time, and task scope.
- Workflow: Tasks simulate the use of AI agents in research—setting objectives, planning methodology, and delegating implementation.
- Results: Evaluating 13 frontier models, the best model reproduced an average of 48% of requested artifacts, suggesting agents are approaching research study-scale capabilities.
Related event: BixBench3: Frontier AI Agents Reproduce 48% of Bio Workflows(4 posts)→
More from Research
- Cohere Labs world model course session 4: DreamerV1-V4 architectures in one go — Cohere_Labs · 2026-10-16
- Reasonable post-trains Nemotron 3.5 on 3,000 Verus proofs, lifting pass@1 ~30 points to 73.3% — fhuszar · 2026-10-16
- GPT-6.1 Sol audit finds 6% of arXiv papers had false main theorems, down to 1.5% recently — nitarshan · 2026-10-16
- One tiny word changes the conversation: how LLMs pick up on 'just' — StellaLisy · 2026-10-16
- SEAR: first large-scale controlled study of self-evolving robot agents, 7 LLMs, 330 tasks — xiye_nlp · 2026-10-16
- After Claude-assisted elliptic curve records fell, EC Rank challenge opens 156 record slots — Jsevillamol · 2026-10-16