Anthropic models most complete, OpenAI models more accurate on medical benchmarks
ArtificialAnlys · x · 2026-10-22
Artificial Analysis compared 14 models with results on both MLCR-AA (the private held-out set of Wisedocs' Medical Long Context Reasoning benchmark) and OpenAI's HealthBench Professional. Finding: Anthropic models give the most complete answers, while OpenAI models are more accurate and concise.
- MLCR-AA tests multi-document synthesis over fragmented medical records of roughly 25,000–64,000 tokens; responses must cover every expert-annotated key point, cite supporting evidence, and pass a majority vote of three LLM judges; the headline metric is pass rate
- The evaluated set covers the two hardest tiers (expert-tier clinical synthesis and compound multi-part reasoning): 60 questions, 3 runs each
Full MLCR-AA and HealthBench Professional leaderboards are public.
Related event: Medical Benchmark Showdown: Anthropic Most Complete, OpenAI More Accurate(2 posts)→
More from Models
- DeepSeek-V4.1-Flash spotted running 5 parallel research subagents at near-zero cost — teortaxesTex · 2026-10-22
- xAI adding 200,000 GB300s a month, 10GW of compute planned next year — gaganghotra_ · 2026-10-22
- Anthropic most complete, OpenAI most accurate: 14 models compared on medical benchmarks — ArtificialAnlys · 2026-10-22
- BitKindle: AI bootstraps a full C compiler from a 3,584-byte machine-code seed — dotey · 2026-10-22
- A tokenization-shared open model ladder is sorely missing, and all roads lead to Qwen — bclavie · 2026-10-22
- Chonk DOOM: a DOOM-like shooter built for Mistral Large 4 to beat on its own — GuillaumeLample · 2026-10-22