Running Qwen 3.8 Flash Next With 130k Context on 16GB VRAM at 15-20 t/s
AvidCyclist250 · reddit · 2026-09-24
After a week of testing, the author shares a full recipe for running Qwen 3.8 Flash Next on an RTX 4080 (16GB VRAM) + 64GB RAM, hitting 15-20 t/s at 130k context.
Four things most people miss: the right quant (AtomicChat AD-4.27bpw / Q4KM target with the shared Unsloth MTP head), the right branch (their pr-mtp-fix branch, i.e. llama.cpp PR #28243 plus one fix commit — plain master can't load this MTP head yet), and the right cache flags (q8 KV cache).
The key trick is --spec-draft-cpu-moe: speculative draft experts live in RAM so the target model's hot experts stay on GPU. Versus 10 t/s with IQ4XS, this setup reaches 16.5 t/s generation / 350 t/s prefill at 131k context. Details in the GitHub repo.
More from Infra
- Ornn: token prices plunge while GPU rents surge, value flows to compute owners — FinanceYF5 · 2026-09-24
- Study: data center buildout 2015-2024 slightly lowered US electricity prices — Afinetheorem · 2026-09-24
- Token prices fell up to 62% since June while B200 rents rose 50% — FinanceYF5 · 2026-09-24
- Lium's decentralized GPU cloud spans 68 datacenters in 21 countries, pays for idle GPU minutes — markjeffrey · 2026-09-24
- Open-source Terraform project spins up maintenance-free Kubernetes on Hetzner, 3.9k stars — BLUECOW009 · 2026-09-24
- Qualcomm, Samsung, Cerebras and d-Matrix all tease compute-stacked 3D-DRAM — AccBalanced · 2026-09-24