Kimi K3 and Fable Share Strikingly Similar Failure Modes
ZainHasan6 · x · 2026-07-18
The chart compares the failure distributions of Kimi K3 max and Fable 5 xhigh, revealing almost identical "failure fingerprints":
- About 65% of failures for both are near misses (very close to the correct answer but falling short).
- Both strongly tend to "protect the baseline," rarely breaking existing test suites.
- The common open-source issue of "breaking the baseline" is not prominent in these models.
- Replies also note a per-task correlation of 0.72 between Kimi K3 and Fable, indicating highly similar behaviors and suggesting the benchmark might be reaching saturation.
More from Models
- DeepSeek cut KV cache per token from 389KB to 890B in two years as open-closed gap debate rages — teortaxesTex · 2026-10-27
- GPT-6 Astra controls dexterous hands zero-shot at just 5.2% success in 500 trials — stingning · 2026-10-27
- Unverified: fable 5.5 speculated at 4-5T parameters to outscale 1-3T Opus — haider1 · 2026-10-27
- Kimi K3 Is 'Open' Until You Succeed: License Caps Free Use at $20M Revenue — alex_verem · 2026-10-27
- Screenshot shows 'GPT-6 Astra Ultra' in ChatGPT model picker — unverified leak or joke — justalexoki · 2026-10-27
- DeepSeek Safety Eval Finds "Over-Attribution": Model Mistakes Ordinary Tasks for Tests — Much_Boysenberry9124 · 2026-10-27