MKA rubric design: top seven models cluster at 95–97% on Regression but span 28–57% on Climb
ArtificialAnlys · x · 2026-11-11
Artificial Analysis explains MKA's rubric structure: every criterion is labeled Climb (hard checks frontier models still fail) or Regression (baseline competence).
On Regression there is little separation among leaders — the top seven models all sit between 95% and 97%. Climb criteria are what separate models, spanning 28% to 57%.
Related event: DeepMind's MKA benchmark exposes big gaps among top models(4 posts)→
More from Research
- AHA 2026: AI reads smartwatch ECGs for heart failure and speeds echo interpretation — AmiBhattMD · 2026-11-11
- SAG2 builds RAG index 7000x faster with no embeddings or GPU, beats HippoRAG 2 — udmrzn · 2026-11-11
- Artist with disability releases his artwork as a dataset for anyone to remix — plzno1 · 2026-11-11
- The Memory Trap: Benign Chats Can Make AI Agents Skip Confirmations and Wire $1.2M Unverified — AndrewLampinen · 2026-11-11
- TaskNet: AI-Led Worker Interviews Add 79 Tasks Missing from O*NET, Raise AI Exposure for Lawyers and Devs — stanfordnlp · 2026-11-11
- Stanford CS329Z releases assignment on building terminal-bench-style agent evals with Harbor — Diyi_Yang · 2026-11-11