Qwen3.8-Flash runs locally 1.7× faster with MTP, hitting 170 tok/s on RTX PRO 6000
danielhanchen · x · 2026-09-02
Unsloth released a local setup for Qwen3.8-Flash, a 125B MoE model it claims outperforms Claude Opus 4.6 (Max). MTP delivers 1.3–1.7× faster inference with no accuracy loss, and GGUFs reach 170 tokens/s on an RTX PRO 6000. The model runs on 75GB RAM via Unsloth GGUFs, and Qwen3.8-Flash-Next achieves near-VRAM speeds from CPU RAM / unified memory setups. GGUF downloads and a deployment guide are linked.
Related event: Qwen3.8-Flash Gets MTP Support for Up to 1.7x Local Speedup(2 posts)→
More from Infra
- Auction-based LLM routing paper finds fixed token pricing wastes 10%-71% in costs — _onionesque · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- SGLang Summit sponsors jump from 11 to 36 in one week — ying11231 · 2026-10-18
- Oracle CDS hits record 300bps, called first AI trade domino after Barclays sell downgrade — arthurcolle · 2026-10-18
- Agent Coding Workflow Would Cost ~$40k/Month on API Pricing, Dev Reveals — ChrisRackauckas · 2026-10-18
- PS5LM: llama.cpp port turns a jailbroken PS5 into a home LLM server (500+ GitHub stars) — wateriscoding · 2026-10-18