Qwen3.8-Flash-Next MTP lands in ik_llama.cpp mainline: 45 to 90 tok/s on RTX 5090, works on 12GB cards
Alternative_Will5974 · reddit · 2026-09-04
- The author's MTP support merged into ikllama.cpp mainline (PR #2369), no fork or patch needed
- The model's built-in 2.6B MTP head drafts tokens from hidden states; code draft acceptance hits 93-99%, prose only 60-65%
- Measured decode speeds: 45 → 90 tok/s on RTX 5090 + 128GB (experts on CPU); 85 → 113 on RTX Pro 6000 for code, but prose regressed 83 → 59
- 12GB 4070 got 9.5 → 12.5; a 3090 replicated it; head-target pairing matters, some combos go net negative
- Caveats: single-slot only (-np 1), --jinja hurts acceptance via default thinking mode
- Full build and server configs included, plus three HF repos with MTP head weights
More from Infra
- Developer Moves AI Agents to a Dedicated Linux Workstation Using Incus System Containers — kevinkern · 2026-10-21
- Nvidia guides 70% growth citing unmet AI demand, while media panics over a $500M Korean neocloud IPO — firstadopter · 2026-10-21
- Pichai: everyone is massively short on AI compute — 'everyone wishes you had two times what you had planned' — firstadopter · 2026-10-21
- Stanford CS336 Lecture 6: fused Triton kernels cut softmax memory traffic ~4x — stanfordnlp · 2026-10-21
- General Compute partners with Positron AI: Atlas memory-centric hardware ships Q1 2027 — aftahi_ai · 2026-10-21
- AI bond issuance hits $419B in 2026, up 90% from all of 2025 — SumitGup · 2026-10-21