Open Models Match GPT-6 on MedMCQA but Trail 17 Points on Hard Clinical Cases
keunwoochoi · x · 2026-10-27
Ortet Health's evaluation shows Kimi K3 scores on par with GPT-6 Astra on MedMCQA, but on the harder MedXpertQA Text clinical benchmark the best open model scores 58.7% versus Gemini 4 Argon's 75.9%. Closed models sometimes refuse to answer most medical questions, making open models more willing research collaborators. Individual patient outcome prediction like survival prediction remains hard for both open and closed models.
More from Models
- GPT-6.1 Terra cuts factual error rate from 22.3% to 4.3%, an ~81% drop — haider1 · 2026-10-27
- GPT-6.1 Terra lands: early users praise speed, value and subagent performance — BenBajarin · 2026-10-27
- Leaked GPT-6.1 Terra benchmarks: cheap but 'boring', still far behind Astra and Claude — Angaisb_ · 2026-10-27
- Artificial Analysis Ships Intelligence Index v4.3.2 With GPT-6.1 Terra Comparison — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra cuts hallucination rate by 50 points while lifting accuracy 2 points — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra scores 48 on AA Coding Agent Index at one-third the cost of GPT-5.6 Terra — ArtificialAnlys · 2026-10-27