Mistral Large 4's Terminal-Bench 41% only holds inside Mistral's own agent
julsimon · x · 2026-11-01
Julien Simon dissects Mistral Large 4's release claims: the Terminal-Bench 4.0 score of 41% was measured inside Mistral's own coding agent "Vibe". In a minimal generic agent, both Artificial Analysis and Vals AI show no gain over the preview — the real number is 25-26%. The open weights also carry a license condition: self-use and end-user products are fine, but serving third parties as an inference platform or consultancy requires a separate commercial license at any scale. The release sells a stack (Studio, API, Forge, Vibe), and the 41% describes that stack — while Vibe has only 5,268 GitHub stars vs 223,729 for Opencode. As he puts it: you can download the model, but not the score.
Related event: Mistral Coding Model Benchmarks Questioned as License Terms Draw Scrutiny(4 posts)→
More from coding & agent
- Anthropic bans Claude Code from recreating existing video games amid viral reverse-engineering trend — Polymarket · 2026-11-01
- Open-source CBS sandboxes local LLM agents at the OS level, default-denying risky tool calls — Luoriyy · 2026-11-01
- Hex pivots early: low-code studios become indefensible once AI writes all code — lak_luster · 2026-11-01
- Outerloop Paper Runs Autoresearch Agents as Untrusted Programs — mengyer · 2026-11-01
- Five-model file-based team only matches a single-model harness; orchestration buys reliability, not intelligence — petrusenko_max · 2026-11-01
- Dev adds chatting AI bots to multiplayer games, doubling as server load tests — chongdashu · 2026-11-01