Kimi K3 and Fable Share Strikingly Similar Failure Modes
ZainHasan6 · x · 2026-07-18
The chart compares the failure distributions of Kimi K3 max and Fable 5 xhigh, revealing almost identical "failure fingerprints":
- About 65% of failures for both are near misses (very close to the correct answer but falling short).
- Both strongly tend to "protect the baseline," rarely breaking existing test suites.
- The common open-source issue of "breaking the baseline" is not prominent in these models.
- Replies also note a per-task correlation of 0.72 between Kimi K3 and Fable, indicating highly similar behaviors and suggesting the benchmark might be reaching saturation.
More from Models
- Developer Building a Unified Leaderboard of All Model Benchmark Scores — airesearch12 · 2026-09-11
- Rumor claims Kimi faked performance by serving Claude; DeepSeek new model surprises in evals — realsohamparekh · 2026-09-11
- GPT-5.6 writes well but is instantly forgettable, user complains — BasedRaddka · 2026-09-11
- Opus Refuses Protein Research Codebase Over 'Safety' Concerns, Dev Considers Rolling His Own — josephdviviano · 2026-09-11
- User Hails Unconfirmed 'DeepSeek 4.1 Flash' as an Inflection Point in LLMs — himanshustwts · 2026-09-11
- Terminal Bench v4: GLM-5.3 Leads at 41.9%, Kimi-K3 Underwhelms at 12.6% — Ok_Warning2146 · 2026-09-11