Qwen3.8 Flash IQ3_S hits 40-50 tok/s at 131k ctx on 12GB VRAM + NVMe
bodhi371 · reddit · 2026-10-17
Reddit user bodhi371 got Qwen3.8-Flash-Next-GSQ-RCO-Abliterated running at IQ3S quantization on just 12GB VRAM + 32GB RAM + NVMe, achieving 40-50 tok/s decode and 1300-1500 tok/s prefill at 131k context (44k thinking budget). The setup uses a custom, heavily modified experimental fork of Strata; the author says it "feels like local Opus in some regards." Recipes and code are open-sourced on GitHub.
More from Infra
- CoreWeave drops 15.5% as hot inflation hits AI's leveraged buildout; TSMC, Anthropic demand stays red-hot — pdamodaran · 2026-10-17
- SF Signs Data Center Moratorium; Investor Kwok Asks "What Are We Even Doing" — kevinakwok · 2026-10-17
- Inferact, from the creators of vLLM, launches controllable frontier inference platform — Xianbao_QIAN · 2026-10-17
- AMD MI355X beats NVIDIA B300 by 24.4% throughput serving Kimi K3, 87% cheaper per token — vllm_project · 2026-10-17
- Anthropic: Claude Code responds 3x faster, Claude API 1.5x faster via gzip and infra cuts — ClaudeDevs · 2026-10-17
- Reddit user pairs Mac mini M5 Pro with old M4 as sub-agent for local coding agents — Calm-Set6265 · 2026-10-17