Running Qwen 3.6 27B on RTX 5090: 40 t/s at 262k Context in llama.cpp

Gargle-Loaf-Spunk · reddit · 2026-08-08

A developer shared their optimized llama.cpp launch arguments for running the Qwen 3.6 27B model (Q6K quantization) on an RTX 5090, specifically tailored for app development tasks.

Performance

General tasks achieve around 80-100 t/s.
Speeds drop to approximately 40 t/s at full 262k context.
The model barely fits into the 5090's VRAM without the vision module, leaving very little room to spare.

Key Parameter Tuning

Batch size: Through bracketing coding tests, -b 512 and -ub 128 yielded the best performance, contrary to smaller values commonly used by others.
Reasoning budget: Set to a high 16384. The author noted that lower values cause the model to get "lost" more frequently, which is hard to quantify via evals but is subjectively more stable.

The setup also enables advanced features like MTP speculative sampling (--spec-type draft-mtp) to maximize performance.

Original post →

Running Qwen 3.6 27B on RTX 5090: 40 t/s at 262k Context in llama.cpp

More from coding & agent