Mistral Large 4 Scores 94% on 1,273 Medical Exam Questions

Maziyar Panahi tested Mistral Large 4 with 1,273 USMLE-style medical questions under a fixed prompt and temperature 0, achieving 94% accuracy. Notably, the model still changed its answers on 44 questions despite temperature 0.

2026-10-31 ~ 2026-10-31 · 2 related posts

• Mistral Large 4 scores 94% on 1,273 USMLE-style questions but flips 44 at temp 0 — MaziyarPanahi · 2026-10-31
• Mistral Large 4 flips 44 of 1,273 medical answers between runs even at temp 0 — MaziyarPanahi · 2026-10-31