Mistral Large 4 flips 44 of 1,273 medical answers between runs even at temp 0
MaziyarPanahi · x · 2026-10-31
Follow-up on the medical exam eval: the model was Mistral Large 4 via Mistral's API, same prompt every run at temperature 0 — yet it still changed its answer on 44 of the questions between runs, highlighting instability even under deterministic sampling.
Related event: Mistral Large 4 Scores 94% on 1,273 Medical Exam Questions(2 posts)→
More from Models
- Early user impression: Fable 5 feels like Opus 3 and Opus 4.5 combined — repligate · 2026-10-31
- GPT 6.1 Terra defended: much faster than Sol at Computer Use tasks — haider1 · 2026-10-31
- Sam Altman says OpenAI is sitting on many unpublished mathematical discoveries — kimmonismus · 2026-10-31
- Dev says Opus 5.5 leads on productivity, keeps GPT Pro subscription anyway — kevinkern · 2026-10-31
- Genie 4 generates a guy saying "ThursdAI" in real time; Gemini 4 Carbon tops the new index — altryne · 2026-10-31
- Codex users report 5-hour limits exhausted in 10 minutes on simple tasks — winer666 · 2026-10-31