MLCR-AA Medical Benchmark Update: Claude and GPT-6 Astra Tie at the Top

Episode 1 · MLCR-AA Medical Long-Context Benchmark Update: Claude Sonnet 5.5 Ties GPT-6 Astra at the Top (2026-10-21, 7 posts)

On October 21, Artificial Analysis released a methodology upgrade and new results for MLCR-AA, the Wisedocs medical long-context reasoning benchmark. On the overall score, Claude Sonnet 5.5 (max with fallback) and GPT-6 Astra (max) tie for first in the 41-point band, but the sub-dimensions diverge sharply, and the two top models differ nearly a hundredfold in per-task cost.

Confirmed

• Methodologically, models receive a system prompt framing the task as medical record review, with questions placed after complete synthetic medical records of 25,000 to 64,000 tokens; they must give a conclusion and supporting record citations in a single paragraph. Answers are scored by a three-model jury, whose agreement rate rose to 69%.
• 60 expert reference answers were manually reviewed and revised.
• Accuracy dimension: OpenAI models sweep the top of the first 27 (the top eight), with GPT-6.1 Sol (max) first at 96.7% and GPT-6 Astra (max) second at 95.6%.
• Completeness dimension: Anthropic models take all top-18 spots across tiers, led by Claude Sonnet 5.5 (max with fallback) at 64.8%; the best non-Anthropic model reaches only 42.2%.
• The overall score requires meeting thresholds on both completeness and accuracy: Claude Opus 5.5 (max with fallback) hits 63.3% completeness but only 37.2% accuracy; GPT-6.1 Sol (max) tops accuracy but is limited on completeness.
• The cost-effectiveness analysis shows the two co-leaders differ nearly a hundredfold in per-task cost.

Why it matters

• The benchmark captures multi-document synthesis and reasoning over real-world medical documentation, and the results show no model dominates both completeness and accuracy—Claude and OpenAI each excel on one.
• The huge per-task cost gap means that in real deployment decisions, "strongest" and "best value" may point to entirely different models.

• Claude Sonnet 5.5 and GPT-6 Astra tie atop updated medical record review leaderboard at 41.7% — ArtificialAnlys · 2026-10-21
• Anthropic sweeps medical benchmark completeness top 18, Sonnet 5.5 leads at 64.8% — ArtificialAnlys · 2026-10-21
• OpenAI models sweep medical QA accuracy top spots, GPT-6.1 Sol hits 96.7% on MLCR-AA — ArtificialAnlys · 2026-10-21
• Claude Sonnet 5.5 and GPT-6 Astra tie at 41.7% on medical benchmark via opposite strengths — ArtificialAnlys · 2026-10-21
• Claude Sonnet 5.5 and GPT-6 Astra tie atop medical long-context leaderboard as cost gaps hit 80x — ArtificialAnlys · 2026-10-21
• MLCR-AA revamps methodology: models prompted as medical reviewers, 3-judge panel agrees on 69% — ArtificialAnlys · 2026-10-21
• MLCR-AA Updated: Benchmarking LLMs on 25k–64k Token Medical Records — ArtificialAnlys · 2026-10-21

Episode 2 · Medical Benchmark Showdown: Anthropic Most Complete, OpenAI More Accurate (2026-10-22, 2 posts)

Artificial Analysis benchmarked 14 models on MLCR-AA and OpenAI HealthBench Professional, finding Anthropic models give more complete answers while OpenAI models are more accurate and concise.

• Anthropic models most complete, OpenAI models more accurate on medical benchmarks — ArtificialAnlys · 2026-10-22
• Anthropic most complete, OpenAI most accurate: 14 models compared on medical benchmarks — ArtificialAnlys · 2026-10-22