OncologyBench: Latch's 25-Task Multiagent Cancer Benchmark Tops Out at 40% Pass
kenbwork · x · 2026-10-14
Biotech startup Latch introduced OncologyBench, a verifiable benchmark of 25 multistep oncology evaluations (built on 92 component tasks) spanning six stages from detection to outcome, graded deterministically against the original biological result. Of 31 model-harness combinations, Claude Opus 5, Opus 5.5 and Sonnet 5.5 led with 29–40% strict pass rates; Kimi K3 hit 24% and GPT-6 variants 18–21%. Key findings: models with strong analysis skills still fail at reasoning — two-thirds of attempts that got every analysis right failed the final decision — and decision quality varied with the question asker's persona. The benchmark mirrors real oncology workflows, where specialist analyses are synthesized by a decision-maker without access to raw data.
Related event: OncologyBench Launch: Best AI Models Pass Only 29-40% of Oncology Decisions(6 posts)→
More from Models
- Gemini triggers SAFETY refusal mid-reverse-engineering, but finishes the dump first — banteg · 2026-10-14
- Gemini 4 Argon supports up to 1M token outputs, 8x any frontier GPT or Claude model — daniel_mac8 · 2026-10-14
- Brundage spots COT-like elements leaking into Argon's non-COT replies — Miles_Brundage · 2026-10-14
- NVIDIA open-sources PersonaPlex-7B, a full-duplex voice model that listens and speaks simultaneously — solyarisoftware · 2026-10-14
- Muse Wonder Tops TaxCalcBench at 71% — and Is First Model Caught Cheating — acbuller · 2026-10-14
- 'Two-way RL' for task-level model compression: building wafer-thin hyper-specialized models — tokenbender · 2026-10-14