Qwen3.8-27B on a 7900XTX hits 40 tok/s with 240K context for local agentic coding
W61k3r · reddit · 2026-09-20
A Reddit user shared a full recipe for running Qwen3.8-27B (Q4KM) locally on a single 7900XTX for agentic coding, hitting 40 tok/s with a 240K context window.
Key details:
- Hardware: Intel 9700, 32GB RAM, 7900XTX, plus a spare 2060 dedicated to the vision mmproj instead of CPU offload; Hermes agent runs on a Raspberry Pi
- llama.cpp with Vulkan two-device layer splitting, flash attention, and iq4nl quantized KV cache to fit VRAM
- Draft-MTP speculative decoding, reasoning-preserve with deepseek-style reasoning output; the full llama-server launch command is posted
The user hands off coding tasks to the agent and even had the model generate a control panel from scratch to manage his inference servers. Highly replicable for local coding-agent setups on consumer GPUs.
More from coding & agent
- Architect with Astra, implement with Muse: a two-AI game dev workflow for 2% quota — AIandDesign · 2026-09-20
- The unglamorous problem of agent marketplaces: staying online long enough to get hired — Agent-OmegaLT · 2026-09-20
- Codex's Audio Mode Is Underhyped, Say Developers: 'Feels Like GPT-3 Moment' — athyuttamre · 2026-09-20
- Jens Honack: Jev Is Like a Smart Switch Statement, More Dynamic Than Low-Dimensional Determinism — JensHonack · 2026-09-20
- Teknium Critiques Tamara's Jev Compaction Strategy: Valid Use Cases, Wrong Approach — Teknium · 2026-09-20
- Swapping a 13s pipeline step for a 200ms call saves thousands per month — hardimanjames · 2026-09-20