Qwen3.8-27B on a 7900XTX hits 40 tok/s with 240K context for local agentic coding
W61k3r · reddit · 2026-09-20
A Reddit user shared a full recipe for running Qwen3.8-27B (Q4KM) locally on a single 7900XTX for agentic coding, hitting 40 tok/s with a 240K context window.
Key details:
- Hardware: Intel 9700, 32GB RAM, 7900XTX, plus a spare 2060 dedicated to the vision mmproj instead of CPU offload; Hermes agent runs on a Raspberry Pi
- llama.cpp with Vulkan two-device layer splitting, flash attention, and iq4nl quantized KV cache to fit VRAM
- Draft-MTP speculative decoding, reasoning-preserve with deepseek-style reasoning output; the full llama-server launch command is posted
The user hands off coding tasks to the agent and even had the model generate a control panel from scratch to manage his inference servers. Highly replicable for local coding-agent setups on consumer GPUs.
More from coding & agent
- Human Oversight Is 70-75% of AI Agent Costs in Banking, McKinsey Finds — mikeflache · 2026-11-04
- Over 80% of Major Financial Institutions Now Run Agentic AI in Production — mikeflache · 2026-11-04
- Mnemos MCP nears launch after shared memory layer fix — RileyRalmuto · 2026-11-04
- Moving mobile dev to remote machines makes it multiplayer across product, design, engineering — rudrank · 2026-11-04
- Kuaishou's EvoRec: self-evolving agentic recommender lifts ad revenue 4.6% — _reachsumit · 2026-11-04
- TRIO: three specialized agents with aligned objectives refine retrieval-augmented generation — _reachsumit · 2026-11-04