RTX 4090 Config for Qwen 2.5 27B: No RAM Spill
gavwhittaker · reddit · 2026-08-18
The author shares a configuration for running Qwen 2.5-27B-UD-Q4KXL.gguf on an RTX 4090 (24GB) and 7800x3D (128GB RAM). This setup achieves a max speed of 134 tps and a minimum of 44 tps without spilling into system RAM.
Key parameters include:
- -ngl 99: Offload all layers to GPU.
- --cache-type-k/v q80: Use Q8 quantization for KV Cache.
- --spec-type draft-mtp: Enable speculate draft mode for acceleration.
- -fa on: Enable Flash Attention.
- --reasoning-format deepseek: Simulate DeepSeek reasoning format.
More from Infra
- Running Qwen 27B at F16: Performance and VRAM Needs — Blues520 · 2026-08-18
- Qwen3.8-27B Benchmarks on M2 Ultra 192GB — planetearth80 · 2026-08-18
- GitNexus Boosts Coding Agent Performance by 30% — ycombinator · 2026-08-18
- Complete Guide to Enabling SageAttention on RDNA4: RX 9070 XT Tested — eloxH1Z1 · 2026-08-18
- Qwen3.8-27B optimization hits 1150 tps on RTX 3090 — iamMess · 2026-08-18
- Fal hosts all Topaz Labs models with 16 enhancement endpoints for media — OdinLovis · 2026-08-18