12 Open-Source Models Take a Medical Exam, All Challenging the Answer Key

A test of 12 top open-source models on a medical exam showed scores between 92.8% and 96.4%, led by Kimi K3, yet all models gave the same answer on one obstetrics question that contradicted the official key, raising questions about the exam itself.

2026-11-03 ~ 2026-11-03 · 2 related posts

• 36 runs, 12 open models, one medical exam question — and all disagree with the answer key — MaziyarPanahi · 2026-11-03
• 12 open models all ace a medical exam; Kimi K3 leads, Qwen3.8 27B hits 94.5% on one GPU — MaziyarPanahi · 2026-11-03