Google DeepMind Launches MKA Benchmark: One Task Cost Claude $68 Across 224 Turns
ArtificialAnlys · x · 2026-11-11
Artificial Analysis partners with Google DeepMind to launch the Multimodal Knowledge Agent (MKA) evaluation, a new benchmark for agentic knowledge work over visually complex files.
- Tests models on engineering schematics, architectural drawings, and clinical scans, requiring deliverables like slide decks, spreadsheets, reports, and marked-up images
- The 10-task public dataset includes auditing a building's plans for door/window mismatches across sections and elevations, reported as a slide deck
- In that example task, Claude Fable 5.2 (max) took 224 turns — viewing images 203 times — costing $68 for a single task
Related event: DeepMind and Artificial Analysis Launch MKA Benchmark for Multimodal Agents(6 posts)→
More from Multimodal
- Krea CEO: fewer one-shot demos, more model-collaboration workflows — araminta_k's Tween & Paint shows how — trq212 · 2026-11-11
- 'Make me the most beautiful video you can': Fable 5.2 demo stuns users — teortaxesTex · 2026-11-11
- GenGate turns open-source PhotoCraft into an AI editor with a ComfyUI bridge for any model — LemonySniket · 2026-11-11
- Bedrock Ricki's uncanny AI character videos keep going viral with 'emotional constipation' bit — Own_Shoulder_1593 · 2026-11-11
- MiniMax H3 image model trends on Hugging Face with ComfyUI single-file release — Comfy-Org · 2026-11-11
- MiniMax H3 Image hands-on: fast plastic-free T2I, but edits and consistency fail — Divineboob · 2026-11-11