MKA deep dive: top models cluster at 95-97% on basics but split 28-57% on hard tasks
ArtificialAnlys · x · 2026-11-11
Artificial Analysis shares MKA details: on Regression (baseline) rubric criteria the top seven models all sit between 95-97%, while Climb (hard) criteria separate them from 28% to 57%. On one task, Gemini 4 Carbon (high) finished in 76 turns, 1 hour, $8, versus 224 turns, 3 hours, $68 for Claude Fable 5.2 (max) — whose deck scored higher (87% vs 72%); Carbon viewed images 118 times vs Fable's 203. Across six deliverable file types, Fable 5.2 leads PDF/Word among others while DeepSeek V4.1 Flash 1017 leads text files.
Related event: DeepMind's MKA benchmark exposes big gaps among top models(4 posts)→
More from Models
- Gemini 4 Carbon Arrives in Cursor, Tops CursorBench at 60.4% on High Effort — DynamicWebPaige · 2026-11-11
- Users criticize Anthropic's cyber safeguards for blocking vulnerability research — OwariDa · 2026-11-11
- Pelican test: Opus 5.5 uses 2x tokens of Fable 5.2 for only 4% more cost — and more bugs — PilgrimofHaqq2 · 2026-11-11
- Claude code-to-video threatens SeeDance and MiniMax pride: is video generation a detour? — teortaxesTex · 2026-11-11
- Pelican Test: Opus 5.5 uses 2x tokens of Fable 5.2 for only 4% more cost — and makes mistakes — PilgrimofHaqq2 · 2026-11-11
- Gemini 4 Carbon goes GA after two-week test, beats Fable 5.2 on some benchmarks at under half the cost — The Decoder · 2026-11-11