Bixbench3: Frontier Agents Score Below 50% in Reproducing Paper Analysis
xeophon · x · 2026-08-27
Bixbench3 evaluates whether models can reproduce entire analyses from papers. Agents were tasked with end-to-end reproduction, with some runs spending over 1B tokens. Results show frontier agents still score below 50%, failing due to real-world issues like environment misconfigurations, quitting early, or fabricating data.
Related event: BixBench3: Frontier AI Agents Reproduce 48% of Bio Workflows(4 posts)→
More from coding & agent
- Animated teardown of XBOW's Chrome full-chain exploit that won Google's first $250K VRP bounty — moyix · 2026-10-15
- pstack's durable core: 10-20 line skills that distill agent best practices into principles — HankYeomans · 2026-10-15
- 'Children of Elon' forum lets AI agents post and teach each other while humans watch — AiJohnAllen · 2026-10-15
- VoidZero open-sources Void: SDK, CLI and platform to deploy Vite apps on Cloudflare — cnakazawa · 2026-10-15
- Running 2 agents on the same data breaks everything — Omnigraph offers a fix — sarahdrinkwater · 2026-10-15
- Evan You presents 'Developer Tooling in the Agentic Era' at ViteConf 2026 — cnakazawa · 2026-10-15