New MKA benchmark: Claude Opus 4.5 tops 21 models at 57% on visual knowledge work
patloeber · x · 2026-11-12
Patricio López partnered with Artificial Analysis to launch the Multimodal Knowledge Agent (MKA) benchmark, based on Google DeepMind's design. It tests whether agents can read visually dense professional files (architectural drawings, schematics, clinical scans) and produce professional deliverables.
- 200 tasks across 10 industry verticals (finance, legal, medical, science, etc.)
- Agents run in a Linux sandbox with code execution, internet access, web search and image-viewing tools, capped at 250 turns
- GPT-6 Luna (medium) serves as the agentic judge; headline metric is Climb WSR
Leaderboard (21 models): Claude Opus 4.5 (52.5 max) leads at 57%, followed by Carbon 4 (high) at 54% and Claude Opus 4.5 (35.5 max) at 53%; GPT-6 Sol (max) 50%; Grok 4.7 (xhigh) 39%, DeepSeek V4 Flash 37%, MiniMax M2.5 Flash 33%, with Step 5 Preview last at 28%.
More from Models
- rasbt coins "Benchmaxxing": making the numbers look better (best optics) — infoxiao · 2026-11-12
- Claim: Claude Opus 5.5 built a horse-riding game demo in two days (unverified) — prasenx · 2026-11-12
- OpenAI engineer opens SEV0 as leaderboard scores fail to save — iansilber · 2026-11-12
- Blogger reruns February's Gemini 3.1 Pro demo against new Gemini 4 Carbon to show 9 months of progress — Angaisb_ · 2026-11-12
- Whatever happened to OpenAI's math-solving demos from October? — iam_indecisive · 2026-11-12
- Canva says its in-house design models cost 40x less than frontier models with 100x fewer parameters — samcharrington · 2026-11-12