Single RTX 5080 runs Qwen3.8-Flash-Next at 150 tok/s decode, 4.4K tok/s prompt
yehyakar · reddit · 2026-10-22
A community benchmark runs Qwen3.8-Flash-Next IQ3XXS with 64K context on a single RTX 5080 16GB (Fedora 44, Strata v0.2.1): 150 tok/s decode and 4,400 tok/s prompt processing at 16K-32K. Key details: LACT-tuned GPU (2850MHz @ 900mV, 360W), 5,310 experts cached in 8.61GiB VRAM, int8 KV with MTP enabled, and a GCC 16/g++-15 linker pitfall on Fedora 44 requiring a unified-compiler rebuild.
More from Infra
- Micron AI memory roadmap: bandwidth moves onto the die, HBM supply tight from 2029 — BenBajarin · 2026-10-22
- Qualcomm's non-phone revenue seen growing 60% next year, driven by auto and embedded — pdamodaran · 2026-10-22
- After PHP advisories, a WebAssembly runtime vendor lays out its security validation plan — MickeySteamboat · 2026-10-22
- Hugging Face: Specializing an SLM's vocabulary can cut inference cost by 77% — Hugging Face Blog · 2026-10-22
- Ollama Runs Open Models on NVIDIA Blackwell via Crusoe, Live in Two Weeks — ollama · 2026-10-22
- CUDA's moat quietly eroding as Codex can rewrite kernels for AMD, investor notes — paigeinsf · 2026-10-22