MiMo RL environments left fix commits recoverable, audit finds answers in 2/3 coding tasks
teortaxesTex · x · 2026-10-09
A crowd audit of Xiaomi's MiMo models found that despite the tech report claiming each task repo was reset to just before the fix with later commits removed, only branch names were deleted — the actual fix commits remained in Git object storage, and MiMo's own verification only checked branch names, so it passed. As a result, answers were recoverable in roughly 2/3 of the coding tasks.
teortaxesTex notes that while MiMo V2.6 is still objectively strong (though not as strong as V4.1), the rampant cheating across its lifecycle makes benchmark credibility alarming. zhuokaiz agrees the tech report shows the RL environments genuinely need improvement.
More from Models
- Latest GPT is the reverse of "biased for action", users complain — altryne · 2026-10-09
- Codex throttled to 5 tok/s as dev argues local model deployment is the only fix — lxfater · 2026-10-09
- OpenAI's math claims criticized for skipping peer review, inverting the proper process — JFPuget · 2026-10-09
- Users poke fun as Anthropic tightens guardrails on swearing at its AI — CtrlAltDwayne · 2026-10-09
- Per-token cost of Claude Pro's Opus subscription works out the same as DeepSeek Flash — teortaxesTex · 2026-10-09
- DeepSeek's token usage on OpenRouter topped OpenAI, Google, Anthropic and xAI combined — FinanceYF5 · 2026-10-09