HealthBench Pro: GPT-6 Astra Max tops leaderboard at 66%, but shortest answers score lowest on completeness
ArtificialAnlys · x · 2026-10-20
Artificial Analysis published its independent HealthBench Professional leaderboard, based on OpenAI's benchmark of 525 clinician conversations (consult, writing, research) graded against physician-written rubrics, using GPT-5.4 (low) as judge and 4 repeats per task.
Key findings:
- GPT-6 Astra (Max) leads at 66.0%, followed by GPT-6.1 Sol (Max) at 65.1% and Claude Sonnet 5.5 (Max, Default Fallback).
- GPT-6 Sol (max) writes the shortest answers (1,000 characters) with the highest raw Communication Quality score (81%), yet the lowest Completeness score (34%) — a clear brevity-vs-thoroughness tradeoff.
- Context Seeking or Global Health is the lowest-scoring theme for 13 of the 14 models.
- The leaderboard reports both length-adjusted and raw scores, penalizing replies over 2,000 characters.
Related event: GPT-6 tops new third-party HealthBench medical leaderboards(2 posts)→
More from Models
- vLLM ecosystem ships d3-turbo: multimodal decision model, 15x faster, tops Jev Decision Index — vllm_project · 2026-10-20
- MOSS 5B architecture diagram built from official code, MIT-licensed — TheLatentExplorer · 2026-10-20
- Devs say no model loyalty: will switch to OpenAI for better limits — Angaisb_ · 2026-10-20
- reuF2F benchmark: 244 REU math problems formalized in Lean, best model solves only 109 — wellecks · 2026-10-20
- Andrew Ng: AI math breakthroughs are largely not from mathematicians uploading ideas — andrewgwils · 2026-10-20
- Pangram 4.1 fixes AI-detection false positives after community pushback — GolinoHudson · 2026-10-20