Anthropic most complete, OpenAI most accurate: 14 models compared on medical benchmarks
ArtificialAnlys · x · 2026-10-22
Artificial Analysis compares 14 models across MLCR-AA (long-medical-record reasoning, grading completeness and accuracy) and OpenAI's HealthBench Professional (physician rubrics, with a length-adjusted score).
- Anthropic leads completeness and raw scores: Claude Sonnet 5.5, Opus 5.5 and Fable 5.1 top MLCR-AA completeness 19-23 points above the best OpenAI model; Opus (71.4%) and Sonnet (70.8%) narrowly beat GPT-6 Astra (69.7%) on HealthBench Professional raw score.
- OpenAI leads accuracy and conciseness: Anthropic models write 4-6x more per answer yet score 19-61% on accuracy vs 89-97% for OpenAI — more claims mean more chances to contradict the record.
- Scores rewarding both close the gap: GPT-6 Astra ties Claude Sonnet 5.5 for #1 on MLCR-AA and narrowly leads the length-adjusted HealthBench score; Opus falls from #1 raw to #6 length-adjusted.
The chart also shows Kimi K3, GLM-5.3 and DeepSeek V4.1 Flash in the mid-lower range.
Related event: Medical Benchmark Showdown: Anthropic Most Complete, OpenAI More Accurate(2 posts)→
More from Models
- DeepSeek-V4.1-Flash spotted running 5 parallel research subagents at near-zero cost — teortaxesTex · 2026-10-22
- xAI adding 200,000 GB300s a month, 10GW of compute planned next year — gaganghotra_ · 2026-10-22
- BitKindle: AI bootstraps a full C compiler from a 3,584-byte machine-code seed — dotey · 2026-10-22
- A tokenization-shared open model ladder is sorely missing, and all roads lead to Qwen — bclavie · 2026-10-22
- Chonk DOOM: a DOOM-like shooter built for Mistral Large 4 to beat on its own — GuillaumeLample · 2026-10-22
- DeepSeek V4.1 Pro rumored to train on 50T+ pretraining tokens, matching Microsoft's scale — teortaxesTex · 2026-10-22