Kimi K3.1 accused of benchmaxxing: audit shows big gaps on fresh internal benchmarks
brainExploded99 · reddit · 2026-11-05
A Reddit post argues Kimi K3.1 is likely benchmaxxed, citing a third-party benchmark audit PDF and a comparison chart. On fresh internal benchmarks like Hy-FrontierMath T5, Goal2Repo and Foundation Model Bench, Kimi K3.1 Pro trails Claude Opus and GPT models by wide margins (e.g. 13.0 vs 91.3/97.5 on Hy-FrontierMath), only matching GPT on BuddyVQA Atlas.
The author's thesis: on brand-new benchmarks Chinese models consistently underperform, then improve in later releases — a sign older benchmarks have leaked into training data. Root causes are structural: the US has a large ecosystem of companies producing and selling training data for frontier labs, plus a compute gap; China lacks the equivalent data industry. The author invites rebuttals with evidence.
More from Models
- Kimi K3.1 Pro (2.8T params, 104B active) lands on Ollama Cloud with zero data retention — ollama · 2026-11-05
- Linear CEO Karri Saarinen: time to retire the word "context" in AI — round · 2026-11-05
- Codex on the $20 plan runs dry after about 2.5-3 hours of nonstop work — tristanbob · 2026-11-05
- Morph launches Reflex: first decision model with cached pricing, 10-20x cheaper than Jev — GabGarrett · 2026-11-05
- Hamel Husain: Benchmarks are used like credit scores, so criticism is warranted — HamelHusain · 2026-11-05
- Reddit spots a possible new Gemini Lite model, first since 3.5 Flash Lite — ygsbhardwaj · 2026-11-05