27B uncensored Qwen at 160K context on one RTX 5090: 140-190 tok/s with DFlash2
Fz1zz · reddit · 2026-09-16
A detailed writeup of running Qwen3.8-27B RVN Heretic (ARA abliterated, Q6K) with tools and vision on a single RTX 5090 at 160K context for long agentic coding sessions.
Measured results (2026-09-16):
- DFlash2 speculative decoding (n=4): 140-190 tok/s on code, 100 on prose; 62 tok/s without speculation
- Cold prefill: 32K in 11.7s, 64K in 28.6s, 113K in 60s
- VRAM: 30 GB after load, 30.3 GB after first image
- Post-quantization KL 0.0085, 0-1/100 refusals
Setup notes:
- Requires merged llama.cpp PRs DFlash2 (#27342) and the image speculation fix (#28715)
- q80 K/V caches; --cache-ram 16384 parks prompt states in system RAM to avoid re-prefill in long conversations
- CUDA graphs must be off (Xid 8 hangs on 5090, #27330); do not add --swa-full
- Full download commands and llama-server launch flags included, ready to copy
More from coding & agent
- Dev builds a "Content OS" on Grok Bot that scouts, researches, and writes his posts — minchoi · 2026-09-17
- Students open-source a no-vector-DB MCP memory server with 30 tools and trust decay — Neither-Witness-6010 · 2026-09-17
- Cognition launches Devin Code Scans: codebase-wide audits that open fix PRs via Agentic MapReduce — DevinAI · 2026-09-17
- After 35 years coding, retiree now lets AI write all his code, only tests and debugs — Sirtemed · 2026-09-17
- Clay built agents on Vercel AI SDK; LangSmith tracing took one line — LangChain · 2026-09-17
- Sentry CEO David Cramer doubts WebMCP's value, clashing with Wes Bos's hype — zeeg · 2026-09-17