36 runs, 12 open models, one medical exam question — and all disagree with the answer key
MaziyarPanahi · x · 2026-11-03
Blogger MaziyarPanahi tested 12 top open models on an obstetrics exam question: a 31-year-old at 32 weeks with ruptured membranes for 26 hours and contractions. The answer key says cesarean section, but all 36 runs (3 per model) uniformly chose induction of labor.
The vignette shows a reactive fetal heart rate, no decelerations, and stable vitals — so the models may be right and the key wrong, or vice versa. The author is asking physicians to weigh in on the discrepancy.
Related event: 12 Open-Source Models Take a Medical Exam, All Challenging the Answer Key(2 posts)→
More from Fun
- X users report a sharp surge in bot replies over the past few days — jaivin · 2026-11-03
- Gatsby founder builds AI-made game Slag, burns through 100% of weekly tokens — zeeg · 2026-11-03
- User flags viral video as 99% likely AI-generated, citing Sora-like artifacts — qtnx_ · 2026-11-03
- User's agent makes an unintended pun about "organising" — "AGI achieved" — geoffwolfe · 2026-11-03
- Claude's Login Gauntlet: SSO, 2FA, WorkOS, 1Password, Duo — Hacked Anyway — vasuman · 2026-11-03
- One Commit Touches 1241 Files, +25066/-9 Lines: A 'Yolo' AI Coding Masterpiece — tomjohndesign · 2026-11-03