Running Qwen3.8-27B for local coding on 16GB VRAM: full setup guide
Due-Project-7507 · reddit · 2026-08-26
The author shares a complete setup for running Qwen3.8-27B (exl3 quantization) with OpenCode on a laptop RTX A5000 (16GB), via exllamav3/tabbyAPI — good enough for small HTML games and web tasks, with the model asking clarifying questions then writing a game in one shot.
Key numbers: 3bpw quant + 6-bit/5-bit KV cache + MTP yields 110k tokens of context; decode speed 55 tok/s for code, 10 tok/s where MTP doesn't help. Dropping MTP, use 3.5 or 4bpw for longer context.
The guide walks through drivers, Git, uv, tabbyAPI install, downloading the exl3 model, swapping in a fixed chattemplate.jinja, and a config.yml with VRAM-saving choices (cachemode 6,5; maxbatchsize 1; MTP draft at Q8; 8192MB sysmem second-tier KV cache), plus a speed-test script.
More from coding & agent
- Dev bills $20,000 for an app built with $200 of Claude Code — IgorCarron · 2026-10-15
- Dan Shipper: run product and labs in parallel, even with a labs team of one — danshipper · 2026-10-15
- Gemini 4 Argon wrote 5,000+ lines of pure Python to build its own ray-tracing 3D engine — aitrendz_xyz · 2026-10-15
- Minecraft fully ported from Java to Rust, file by file, server and client included — Vjeux · 2026-10-15
- mitsuhiko builds a full Python IDE for iOS in an afternoon with new AI models — mitsuhiko · 2026-10-15
- How to get Gemini 4 Argon to build games that actually look decent — chongdashu · 2026-10-15