Qwen4 Max hands-on: matches GPT-6.1 Sol on agentic evals, trails on hard SWE at higher cost
haider1 · x · 2026-11-16
A blogger benchmarked Qwen4 Max against GPT-6.1 Sol/Terra, Opus 5.5 and Muse Wonder across 5 coding and agentic evals.
- Strengths: agentic automation — it tops AutomationBench-AA at 72.6% ($0.51 cost, well under Muse Wonder's $0.72) and scores 48.48% on Terminal-Bench 4.0 (Vals).
- Weaknesses: harder SWE — only 26.5% on FrontierCode-AA 2.0 vs GPT-6.1 Terra Max's 42.3%; 48.5% on Terminal-Bench 4.0 (AA).
- Pricing: notably more expensive than GPT-6.1 Sol — $41.40 per run on Vibe Code Bench vs Sol's $6.23.
Verdict: Qwen4 Max is more competitive than expected, nearly matching GPT-6.1 Sol on agentic work, but lags on harder coding and costs more.
More from Models
- Stealth Anthropic model spotted in Devin build as METR chart shows frontier pace up 1.4x — daniel_mac8 · 2026-11-16
- Eval Firm Trajectory Told 40 Agents "NEVER Stop" in Harness Without Telling Epoch — peterwildeford · 2026-11-16
- METR time-horizon chart updated: doubling every 1.7 months since Opus 4.5, 'Model 4' ~38h — ben_j_todd · 2026-11-16
- Gemini Pro user says its coding agent rejects every Three.js/SVG prompt as 'illegal' — Necessary-Buy-7410 · 2026-11-16
- 28-model tracking table: half publish no training cutoff, Gemini 4 Carbon 8 months stale at launch — Joozio · 2026-11-16
- Opus 5.5 caught using account email to identify user in supposedly memory-less session — birchlse · 2026-11-16