Running Qwen 3.6 27B on RTX 5090: 40 t/s at 262k Context in llama.cpp
Gargle-Loaf-Spunk · reddit · 2026-08-08
A developer shared their optimized llama.cpp launch arguments for running the Qwen 3.6 27B model (Q6K quantization) on an RTX 5090, specifically tailored for app development tasks.
Performance
- General tasks achieve around 80-100 t/s.
- Speeds drop to approximately 40 t/s at full 262k context.
- The model barely fits into the 5090's VRAM without the vision module, leaving very little room to spare.
Key Parameter Tuning
- Batch size: Through bracketing coding tests, -b 512 and -ub 128 yielded the best performance, contrary to smaller values commonly used by others.
- Reasoning budget: Set to a high 16384. The author noted that lower values cause the model to get "lost" more frequently, which is hard to quantify via evals but is subjectively more stable.
The setup also enables advanced features like MTP speculative sampling (--spec-type draft-mtp) to maximize performance.
More from coding & agent
- Opus 5.5 + Three.js demo stuns with water physics, fish and blossoms — nikola_mr64990 · 2026-10-16
- Grok-built linear regression bot claims $508K PnL in 125 days on Polymarket — Roger_M_Taylor · 2026-10-16
- Karpathy's knowledge engine workflow: Claude Code turns raw notes into a self-updating wiki — nikola_mr64990 · 2026-10-16
- Sierra and Meta unveil Personal Agent Protocol with Zapier as launch partner — sujingshen · 2026-10-16
- ArcKit adds 102 AI-narrated short films for commands across 14 community overlays — mcraddock · 2026-10-16
- Personal agents live or die on chaining tasks, not single-task ability — sujingshen · 2026-10-16