GPT-6 Astra tops surgical AI leaderboard but loses to models 1000x smaller
ddonoho · x · 2026-09-06
Researchers benchmarked GPT-6 Astra on the Surgical AI Leaderboard: it is now the top generalist model, yet still falls behind tiny specialist models roughly 1000x smaller in the surgical domain.
A related quoted benchmark of Claude Fable 5.1 and Gemini 3.8 Flash found their performance spiky — strong on tool use, weak on VQA — with frontier LLMs overall still underperforming small specialized models. Full leaderboard and paper linked in the original post.
More from Models
- 105 Real Bugs Tested: GPT-6.1 Sol Over 10x Cheaper Than GPT-5.6 Sol — PawelHuryn · 2026-10-23
- Abliterated V4.1-Flash eval: abstract algebra drops 6pp but college math gains 5pp — teortaxesTex · 2026-10-23
- Claude users hit "model at capacity" errors hourly, canceling plans over it — Alex-Stack · 2026-10-23
- Kimi K3.1 launch imminent: 3T-param bet on RL seen as make-or-break — teortaxesTex · 2026-10-23
- Uncensored fine-tunes vanish from Hugging Face amid unexplained takedowns — umarmnaq · 2026-10-23
- Claude app menu leaks mystery model name 'Fable' as user hits limits — ezshine · 2026-10-23