Running Qwen3 27B at 262k context on a single 16GB RX 9060 XT at 42.7 tps
Maglcite · reddit · 2026-10-18
A Reddit user shows local LLM setups don't need multi-GPU rigs: a single RX 9060 XT 16GB plus 24GB DDR5 runs Qwen3 27B (unsloth IQ4XS, KV cache quantized q4 values / q8 keys) at 262k context using MTP + ngram prediction and KV cache streaming with a llama.cpp fork.
- Performance: 42.7 tps generation, 658 tps prefill at 20k context; 35/300 tps at 200k
- The author even had Grok bot SSH in to fix missing PCIe atomics bugs
More from Infra
- OpenAI reportedly handled ~15T tokens daily in April, likely higher now — JoshPurtell · 2026-10-18
- 1.68x inference speedup drops, directly accelerating GRPO post-training — yacineMTB · 2026-10-18
- OpenRouter is only a fraction of the total addressable market, poster argues — sloppenheimer · 2026-10-18
- Developer builds change alerts for LLM prices and retirements across Bedrock, Azure and Vertex — jsandhu0780 · 2026-10-18
- 12.3M tokens in 27 minutes for $0.14: agent session with 99.6% cache hit — teortaxesTex · 2026-10-18
- SlateDB+Lance+Datafusion stack saves customer millions, as AI agents unbundle big data systems — sh_reya · 2026-10-18