OncologyBench: 25 verifiable oncology evaluations benchmark multi-agent systems
kenbwork · x · 2026-10-14
Researchers have released OncologyBench, a verifiable benchmark of 25 multistep oncology evaluations built from 92 component tasks, spanning six stages of the oncology journey: translational research, risk and detection, diagnosis and tumor characterization, treatment, and beyond. Multi-agent systems must produce clinically relevant decisions, with a final agent synthesizing analytical findings.
Scoring is strict—a run passes only when both tasks and decisions fully pass, with refusals and timeouts left in the denominator—and results are reported per model + harness combination, since the harness substantially changes scores. Claude Opus 5 with the Pi harness tops the leaderboard at 40.0%, followed by Claude Opus 5.5 + Claude Code at 37.3%; Kimi K3, GPT-6 variants, and Grok 4.6/4.7 cluster around 20%. Paper, blog, GitHub repo, and a live leaderboard are available.
Even the best multi-agent setups pass only 40% under strict scoring, leaving a large gap to clinically usable oncology decision-making.
Related event: OncologyBench Released: Top Models Pass Only 29-40% of Oncology Decisions(7 posts)→
More from Research
- Economist compiles 100+ free social science datasets on GitHub — _akpiper · 2026-10-14
- New ISQ article asks why policymakers wait to regulate disruptive tech like AI — mchorowitz · 2026-10-14
- MIT professor's ten predictions for AI for science, from executable papers to compiled matter — ProfBuehlerMIT · 2026-10-14
- Sweden spends hundreds of millions of kronor on printed textbooks as 2.8M-child study links screens to lower grades — Eric520CC · 2026-10-14
- Self-evolving red-team search finds 1.77x more harmful tool-call paths in personal agents — lulzxdxdxd · 2026-10-14
- Google researcher: general capabilities plus tool use are displacing domain-specific medical AI — davidstutz92 · 2026-10-14