Mistral Claims Best Open Coding Duo, But Its Own Cited Benchmark Ranks Kimi 8 Points Ahead
julsimon · x · 2026-11-20
Julien Simon dissects Mistral's Nov 17 claim that Vibe + Le Chonk (Large 4) is the "world's best open model-harness coding duo." The benchmark index Mistral's own release page cites (Artificial Analysis) has ranked Kimi 8 points ahead since Nov 14 — and the post lists rivals while skipping the one that leads.
- Mistral's "evidence" was a collage of seven fan posts: no numbers, no chart, no benchmark link
- The copywriting is selectively generous: a small deficit is "effectively tied," a small lead is "a lead," a slightly bigger one is "actually better"
- Mistral's benchmark numbers have repeatedly been walked back recently (an unnamed-agent preview score, an AA-deleted-then-restored headline run)
- The honest pitch was right there: same score as rivals at a fraction of the cost; three free months in Europe is a good offer that needs no fact-twisting
More from Fun
- antirez's Polar Sensor Got Bricked by a Firmware Update — Claude Fixed It Over USB and Bluetooth — antirez · 2026-11-20
- Pedro Domingos quips: who needs funhouse mirrors when we have LLMs — pmddomingos · 2026-11-20
- A hobby podcast ended up in Pearson's official AS English Language exam — tinyfool · 2026-11-20
- antirez: Big Manufacturers' Bad Software Is a Structural Failure, Not a Coding One — antirez · 2026-11-20
- Anthropic employees vanish from LinkedIn, headcount graph shows -99% — basedjensen · 2026-11-20
- Researcher jokes her husband called passing on a now-$23B billionaire "not wise" — basedjensen · 2026-11-20