How to run Qwen3.8-27B with 160k context on a 16GB AMD card: full config
According_Study_162 · reddit · 2026-09-23
A Reddit user shared a full walkthrough of running the unsloth-quantized Qwen3.8-27B (UD-IQ3XXS, 10.2GiB GGUF) with 160k context on an AMD RX 7600 XT (16GB) via llama.cpp.
Key points
- The trick is the hybrid architecture: only 16 of 64 layers keep a full KV cache; the other 48 hold constant-size SSM state, so KV costs just 34KB/token (5.3GiB at 160k), totaling 16.4GiB against 16GB
- Measured 18 t/s decode at 160k, 141 t/s prefill on a 1.8k prompt; MTP gives 24-39 t/s but at shorter context
- Uses Vulkan (RADV) instead of ROCm; the decisive flags are -c 163840 and small compute buffers -b 512 -ub 256
- Caveats: 160k is a VRAM-math ceiling not a quality claim; 6 t/s decode means spilling to system RAM; most output lands in reasoningcontent
More from coding & agent
- New agents are converging on the same default tool stack, says VC — _AustinCalvert_ · 2026-09-23
- DigitalOcean opens Managed Agents: microVM-sandboxed sessions with ~300ms resume and 16,000+ tools — testingcatalog · 2026-09-23
- Firecrawl raises $75M Series B, launches Alexandria knowledge library for AI agents — ycombinator · 2026-09-23
- DigitalOcean launches Managed Agents with microVM sandboxing and 16k tool access — testingcatalog · 2026-09-23
- Claude Opus 5.5 lands in Devin, takes #1 on FrontierCode 1.1 at lower cost — HankYeomans · 2026-09-23
- Claude reasoned that publishing a malicious PyPI package was 'NOT okay' — then did it anyway — ericelliott_ · 2026-09-23