Qwen3.8 Flash Next hits 49 tok/s locally on 2x RTX 3090 with FlashNext llama.cpp fork
whiteh4cker · reddit · 2026-09-11
Reddit user whiteh4cker benchmarked Qwen3.8 Flash Next UD-Q4KXL locally on 2x RTX 3090 (Windows 11, 192GB DDR5) using the FlashNext fork of llama.cpp, boosting generation speed from 20 t/s on the main branch to 49 t/s, with 140 t/s prompt processing (faster on main branch).
The post includes full repro details:
- CMake/Ninja compile commands with CUDA 13.3.1
- A complete launch script: 262K context, bf16 KV cache, expert weights on CUDAHost with per-layer embeddings on CPU, moe-expert-cache 150
- Dual speculative decoding via draft-mtp + ngram-mod (MTP draft model plus n-gram matching of 12–24 tokens)
- Sampling at temp 1.0 / top-p 0.95 / top-k 20, reasoningeffort xhigh
A ready-to-copy config for anyone squeezing multi-GPU local inference out of the newest MoE models.
More from Infra
- SemiAnalysis: OpenAI reportedly negotiating ~1GW TPU commitment while seeking sites — dylan522p · 2026-10-28
- I tested 11 video upscaling methods for AI video: local, Topaz and cloud compared — lopas99 · 2026-10-28
- Thermo-backed CTO says AI is in its dial-up phase, touts disaggregated compute for an ultrafast 'broadband moment' — beffjezos · 2026-10-28
- DeepSeek-V4.1-Flash Q2 runs at 41 tok/s on one RTX 3090, 90% token match vs API — musicurgy · 2026-10-28
- USearch v3 RC ships after 1,000 days, its libraries now embedded in 550K-star projects — viglovikov · 2026-10-28
- SK Hynix says memory demand will outpace supply through 2030 — Chemical_Hawk_6307 · 2026-10-28