MKA benchmark results: top models hit 28-57% on hard agentic visual tasks
ArtificialAnlys · x · 2026-11-11
Artificial Analysis published its implementation of Google DeepMind's Multimodal Knowledge Agent (MKA) benchmark, testing whether agents can read visually dense professional files (architectural drawings, schematics, clinical scans) and produce real deliverables across 200 tasks in 10 verticals. Agents run in a Linux sandbox with code execution, web access and image tools, capped at 250 turns, judged by GPT-6 Luna (medium) with weighted rubric criteria. Key finding: on Regression (baseline) criteria the top seven models cluster at 95-97%, but Climb (hard) criteria separate them, ranging from 28% to 57%.
Related event: DeepMind's MKA benchmark exposes big gaps among top models(4 posts)→
More from Models
- TheZmi flags Fable 5.2 regression: actively less robust across model card benchmarks — TheZvi · 2026-11-11
- Siri AI's Skimmer: regex-only fast path calls tools up to 20× faster than LLMs — SumitGup · 2026-11-11
- Stripe CEO: Higher Reasoning Effort Thinks Harder Per Option, Not About More Options — gabriel1 · 2026-11-11
- Poll: 70% say Anthropic's best internal model plus $50M can dominate any human expert — JacquesThibs · 2026-11-11
- Ollama cloud 'DeepSeek' claimed it was actually ChatGPT, user alleges model swapping — No_Literature_7653 · 2026-11-11
- Rumor: Anthropic's Claude 4 said to be 'very different' in cyber and RSI capabilities — ChrisGPT · 2026-11-11