New MKA Benchmark: 200 Expert-Level Multimodal Tasks Across 83 Occupations
osanseviero · x · 2026-11-17
Google DeepMind and Artificial Analysis released the Multimodal Knowledge Agent (MKA) benchmark, now open on Hugging Face: 200 tasks across 83 occupations (3–8 hours of expert work each), going beyond OCR with blueprints, schemas and charts, requiring agents to produce decks, docs and sheets, with an open harness and sample tasks.
Related event: Google and Artificial Analysis Launch MKA Benchmark for Multimodal Agents(2 posts)→
More from Models
- Model-picker screenshots surface unannounced GPT-6.1, Gemini 4 Hydrogen and Claude Opus 5.5 names — TomLikesRobots · 2026-11-17
- OpenAI Brings in Apollo, METR, Transluce and Goodfire for Embedded Safety Assessments — OpenAI · 2026-11-17
- MKA Benchmark: Gemini 4 Carbon Scores 54.3% at $5.63 Per Task vs Claude's $35.28 — _philschmid · 2026-11-17
- Gemini Decisions API Judges 1,000 Questions Over 256K Tokens With Zero Output Token Billing — _philschmid · 2026-11-17
- RWKV-8 RPU preview hits 12,500+ tokens/s real-time inference — kastnerkyle · 2026-11-17
- Gemini 4 Helix spotted in Google Antigravity, hinting at another Gemini 4 variant — feblet · 2026-11-16