27B uncensored Qwen at 160K context on one RTX 5090: 140-190 tok/s with DFlash2
Fz1zz · reddit · 2026-09-16
A detailed writeup of running Qwen3.8-27B RVN Heretic (ARA abliterated, Q6K) with tools and vision on a single RTX 5090 at 160K context for long agentic coding sessions.
Measured results (2026-09-16):
- DFlash2 speculative decoding (n=4): 140-190 tok/s on code, 100 on prose; 62 tok/s without speculation
- Cold prefill: 32K in 11.7s, 64K in 28.6s, 113K in 60s
- VRAM: 30 GB after load, 30.3 GB after first image
- Post-quantization KL 0.0085, 0-1/100 refusals
Setup notes:
- Requires merged llama.cpp PRs DFlash2 (#27342) and the image speculation fix (#28715)
- q80 K/V caches; --cache-ram 16384 parks prompt states in system RAM to avoid re-prefill in long conversations
- CUDA graphs must be off (Xid 8 hangs on 5090, #27330); do not add --swa-full
- Full download commands and llama-server launch flags included, ready to copy
More from coding & agent
- Microsoft's MemSteer turns unlabeled user histories into agent memory supervision — MicrosoftResearch · 2026-11-02
- She fired herself from her own Instagram: AI agent runs the account, views up 82.3% — heyshrutimishra · 2026-11-02
- Sourcegraph: AI writes code faster than teams can review it, driving more outages — AI Engineer · 2026-11-02
- Tsinghua & Microsoft's HarnessMM cuts agent token use by up to 87.93% while raising success rate — rohanpaul_ai · 2026-11-02
- GitHub launches Agent HQ: Anthropic, OpenAI, Google coding agents in Copilot — emmanuelvivier · 2026-11-02
- DeepSeek open-sources Super-Agent, a coding agent that reads team chats to minimize human input — emmanuelvivier · 2026-11-02