Mistral touts STEM benchmark win, but leaderboard shows it ranking sixth overall
MistralAI · x · 2026-10-16
Mistral's official account highlighted Mistral Large 4's engineering and astronomy scores on Ulam's private STEMBench, but the full leaderboard tells a different story:
- GPT-6 Astra xhigh (with tools) leads at 2.809, followed by GPT-6.1 Sol at 2.759, Muse Wonder 2.683, and Claude Opus 5.5 2.621.
- Mistral Large 4 averages 2.501 — sixth of ten runs — with its best marks in engineering (2.602) and astronomy (2.597).
- DeepSeek V4.1 Flash scores 2.490, Gemini 4 Argon 2.444, Qwen3.8-27B 2.314, and MiMo V2.6 Pro trails at 2.172 (just 0.983 in physics).
- STEMBench spans six STEM domains (20 tasks each, 0–4 progress scale), and none of the ten runs achieved a single full-target success.
More from Fun
- Cybertruck rocket-hover meme: 'Sir, license and registration...' — evilsocket · 2026-10-16
- Musk shows off a flipping flying car demo: "the future does have flying cars" — TinfoilTricorn · 2026-10-16
- Dev builds oddly satisfying self-fixing sand component that repairs itself — prasenx · 2026-10-16
- Flying cars are finally real, and the singularity hype feels earned — rand_longevity · 2026-10-16
- Complex multi-link CAD mechanism finally animates, dev celebrates the breakthrough — zetalyrae · 2026-10-16
- Engineer jokes that talking to his coding agent in Hindi revealed 'true free will' — serious_mehta · 2026-10-16