Running Qwen3.8 27B on 16GB VRAM: IQ3 quant hits 131k context at 9-30 tps

randomgenericbot · reddit · 2026-10-07

A detailed hands-on guide to running Qwen3.8 27B on a 16GB GPU (5060Ti, single-channel DDR5). The short answer: it works, with caveats.

Key points:

Use the GSQ-RCO IQ3S quant from ISTA-DASLab; the author benchmarked it side-by-side against a hosted Q8 on an H200 and couldn't tell them apart except for speed
For large context you must build a custom llama.cpp fork with adaptive KV streaming; this forfeits MTP, which only added 5tps anyway
Measured: 30tps at empty context, dropping to 9-10tps near 131k context; prefill declines steadily (880tps at 14k tokens, 630tps at 94k)
Tuning: a 1.5G KV window is a sweet spot (30k context fully in VRAM, headroom for other GPU tasks); Q8 KV cache gets 32k-68k context on stock llama
Practical path: load the model on stock llama, use a harness like pi.dev to have the model itself help compile the fork — copy-pasting console output in chat mode is enough, no kernel-hacking required

More from Infra