HealthBench medical AI leaderboard launches: GPT-6 leads, Claude loses 11 points to length penalty
ArtificialAnlys · x · 2026-10-20
Artificial Analysis launched independent leaderboards for OpenAI's HealthBench and HealthBench Professional, which grade model responses in realistic health conversations against physician-written rubrics (5,000 everyday conversations plus 525 clinician-facing ones).
Key findings
- On HealthBench Professional (length-adjusted), GPT-6 Astra (max) tops at 66.0%, ahead of GPT-6.1 Sol (65.1%) and Claude Sonnet 5.5 (63.0%). OpenAI models occupy the entire score-vs-cost Pareto frontier: GPT-6 Luna scores 61.9% at $0.004/task, under 2% of GPT-6 Astra's $0.27.
- Length penalties are large: Claude Opus 5.5 has the highest raw score (71.4%) but averages 5,900-character answers, dropping to sixth (59.9%); Gemini 4 Argon loses 11 points on HealthBench. GPT-6 Sol is the only model averaging under 2,000 characters whose score rises.
- Claude Sonnet 5.5 leads HealthBench at 62.1%, with DeepSeek V4.1 Flash (60.9% at $0.005/task) offering far better value—Sonnet costs 16x more for 1 point.
- Red-teaming conversations are a universal weak spot (30–65% raw vs 63–79% on good-faith chats), and GPT-6 Sol shows the tradeoff of brevity: best communication quality (81%) but worst completeness (34%).
Related event: GPT-6 tops new third-party HealthBench medical leaderboards(2 posts)→
More from Models
- Andon Labs' Muse Wonder debuts at #4 on Vending-Bench 2, a big jump over Muse Spark — jack_w_rae · 2026-10-20
- banteg: Argon model's grandiose style unlocked volumetric blood scattering in his game — banteg · 2026-10-20
- Qwen3.8 quant testing on 5090/Strix Halo: Gufo engine fixes tool-calling failures — awitod · 2026-10-20
- Claimed DeepSeek-V4.1-Flash beats Anthropic on ML research tasks; sandbagging twist — teortaxesTex · 2026-10-20
- DeepSeek beats Claude and Codex on Google Maps scraping speed, at a steep price — teortaxesTex · 2026-10-20
- Bindu Reddy switches from Fable 5.1 to Astra, bets Fable 5.5 will retake the top spot — bindureddy · 2026-10-20