Qwen2.5-72B local performance rivals Claude 3.5 Sonnet
AGI Hunt · wechat · 2026-08-15
AGI Hunt reviews the Qwen2.5-72B model (referred to as Qwen-3.8-27B in the text), claiming that with just 16GB of VRAM, users can run a model locally that performs close to Claude 3.5 Sonnet (referred to as Opus 4.6 Max).
Key Benchmarks:
- SWE-bench Pro: 61.7 (surpasses Opus 4.6 Max)
- Qwen SWE-Bench: 79.0 (surpasses Opus 4.6 Max)
- TerminalBench: 73.0 (close)
- NL2Repo-Bench: 42.3 (close)
Deployment Advice:
- 4-bit quantization requires 16GB RAM, with speeds around 25-40 tok/s.
- Recommended tools for Mac: MLX/LMStudio/llama.cpp; for PC: vLLM.
The author also mentions Meta's Llama 3.1 405B release and predicts that running Fable5 (GPT-5 level) models on laptops will be possible within two years.
Related event: Qwen2.5 Local Performance Review(2 posts)→
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16