Qwen 3.8 Flash Next q2_0 hits 10 tok/s on a 6GB VRAM laptop via Strata engine
dampflokfreund · reddit · 2026-10-03
- Redditor dampflokfreund ran Qwen 3.8 Flash Next q20 (120B params) on an RTX 2060 laptop (32GB RAM + 6GB VRAM) using the Strata engine
- Result: 10 tok/s decode at 50K context and 100 tok/s prefill — far beyond expectations for such hardware
- Command used: ./START-HERE.bat --draft-vocab en --vram-reserve-mib 100 --kv q40
- Tradeoffs: 4-bit KV cache, no vision; prefill latency remains the bottleneck for long prompts
- Compared to Qwen A35B A3B (5x faster prefill, 100K context without KV quant), it's not a full replacement yet, but an impressive showing
More from Infra
- Tencent leaned on Japanese neocloud for AI compute; DeepSeek, Zhipu best positioned — teortaxesTex · 2026-11-19
- Open-source prefill engine for GLM-5.3-Flash beats vLLM by 3.2x on 4x B200 — CShorten30 · 2026-11-19
- Costco sees panic buying of generators after string of tech outages, 21 of 26 sold out — harris_edouard · 2026-11-19
- Starlink direct-to-cell goes live in Mongolia with free first month — XFreeze · 2026-11-19
- SpaceX unveils $50B, 6GW Arizona compute campus with 24 GWh grid-battery strategy — JOBhakdi · 2026-11-19
- CoreWeave corrects debt rate to 15.5% as analyst presses Blackwell-funded payoff plans — Svwang1 · 2026-11-19