Qwen4 Max per-eval breakdown: strong on knowledge work, lagging in coding
ArtificialAnlys · x · 2026-11-16
Artificial Analysis publishes per-evaluation charts for Qwen4 Max across all 9 Intelligence Index evals. Highlights: AA-Briefcase (agentic knowledge work on real business files) at 1686 Elo, near the top; AA-Omniscience accuracy 44% with markedly improved hallucination rate; AutomationBench-AA 73%. Weak spots remain: Terminal-Bench 4.0 at 48%, Terminal-Bench-Science 22%, FrontierCode 2.0 26% — coding and terminal tasks still trail frontier leaders. HLE-Diamond 45%, CritPt v2 43%, GDP.pdf 29%.
More from Models
- Qwen4 Max scores 43 on AA Intelligence Index, 67% cheaper per task than predecessor — ArtificialAnlys · 2026-11-16
- Qwen4-Max debuts at #14 on Agent Arena, making Alibaba the #5 lab overall — Alibaba_Qwen · 2026-11-16
- New MoE base claims win over Qwen3.8-Max with 50% fewer tokens, met with skepticism — teortaxesTex · 2026-11-16
- Qwen4-Max benchmarks leak: 50% fewer tokens, 60% cheaper — but 'you've hit a wall' — teortaxesTex · 2026-11-16
- Comparing AI censorship: Gemini preempts, Claude silently rewrites, OpenAI just blocks — ivan_bezdomny · 2026-11-16
- Qwen4-Max lands #6 on Code Arena WebDev, up 74 pts over Qwen3.8-Max — Alibaba_Qwen · 2026-11-16