Running Qwen3.8-27B for local coding on 16GB VRAM: full setup guide
Due-Project-7507 · reddit · 2026-08-26
The author shares a complete setup for running Qwen3.8-27B (exl3 quantization) with OpenCode on a laptop RTX A5000 (16GB), via exllamav3/tabbyAPI — good enough for small HTML games and web tasks, with the model asking clarifying questions then writing a game in one shot.
Key numbers: 3bpw quant + 6-bit/5-bit KV cache + MTP yields 110k tokens of context; decode speed 55 tok/s for code, 10 tok/s where MTP doesn't help. Dropping MTP, use 3.5 or 4bpw for longer context.
The guide walks through drivers, Git, uv, tabbyAPI install, downloading the exl3 model, swapping in a fixed chattemplate.jinja, and a config.yml with VRAM-saving choices (cachemode 6,5; maxbatchsize 1; MTP draft at Q8; 8192MB sysmem second-tier KV cache), plus a speed-test script.
More from coding & agent
- OpenAI's Kepler Agent Processes 580PB Daily, Cuts Query Time to 90s — Al_Grigor · 2026-08-26
- AI Dev Tools Zoomcamp 2026 Launches: Free Course on Agents & MCP — Al_Grigor · 2026-08-26
- Developer shares experience of vibecoding a new agent orchestrator app — willcb · 2026-08-26
- xAI launches Grok Bot: AI colleagues that use your tools — tetsuoai · 2026-08-26
- Dev builds Conduit: a browser-control extension for any agent, not just Claude — PumpkinNarrow6339 · 2026-08-26
- CommerceAgentBench Launch: Testing If AI Agents Can Complete Real E-commerce Tasks — alifcoder · 2026-08-26