Running 69GB Qwen3.8 Next on 16GB VRAM + 32GB RAM: full config breakdown
ironicstatistic · reddit · 2026-09-14
A detailed walkthrough of running the 69GB Qwen3.8-Flash-Next REAP pruned variant (320 of 512 experts) on an RTX 5060 Ti 16GB + Ryzen 7840HS + 32GB DDR5 setup.
Key techniques:
- The 29.5GB N-gram/PLE embedding is a pure lookup table and can be streamed via lazy mmap from SSD without hurting speed — the crucial trick that makes the model fit
- The remaining 39.4GB (experts, attention, KV cache) is forced fully into RAM+VRAM with --load-mode none, since OS page-cache mmap yields 2 tok/s
- --n-cpu-moe 34 is the exact tipping point for 64k context: more causes GPU OOM, less crashes the OS via swap thrashing
- Keep ≥2GB RAM for the OS; partial per-layer offload via tensor overrides can squeeze past OOM edge cases
More from Infra
- Your AI agent would cost 260% more without prompt caching, Arena data shows — arena · 2026-10-30
- Nebius runs auction-driven GPU pricing, lifting rates ~20% amid tight supply to 2028 — BenBajarin · 2026-10-30
- macOS update silently re-adds eMudhra root certs users had removed — steipete · 2026-10-30
- Bittensor's Terza Inference routes one API across miners' models on subnet 103 — const_reborn · 2026-10-30
- SGLang Summit 2026 sells out sponsor booths and Full Passes ahead of Nov 12–13 event — ying11231 · 2026-10-30
- OpenRouter rebuilds provider routing: 46% fewer capacity errors, 16.9% faster TTFT — steipete · 2026-10-30