Qwen 27B on RTX 5090: Q4 and Q5 quants both hit 110-120 tokens/sec
sammyranks · reddit · 2026-11-09
A user benchmarked Qwen 3.8 27B (Unsloth GGUF) Q4KM vs Q5KM quants on an RTX 5090 with a one-shot "pelican riding a bicycle" test in OpenCode 2 + LM Studio.
Both quants ran at 110-120 tokens/sec with 31GB VRAM (model + KV cache). Q4KM allowed 120K context vs 90K for Q5KM; MTP on, thinking medium, everything else default.
More from Infra
- Batch APIs: the 50% LLM discount only pays off after model choice and prompt caching — arpit_bhayani · 2026-11-13
- RTX Pro 6000 bricks mid-Qwen run, owner jokes it "died of a broken heart" over Jensen news — wine_dark · 2026-11-13
- Bittensor's Chutes Handles 256 Concurrent Requests on a Single RTX 5090 at ~18k tok/s — markjeffrey · 2026-11-13
- Google expands TPU v8 lineup: 8tx cuts latency 2x, 8i Max doubles silicon per rack — Beth_Kindig · 2026-11-13
- Horace He to share two mental models for reasoning about inference at SGLang Summit — cHHillee · 2026-11-13
- Used A100 prices fall another 20-30% as sellers rush ahead of a collapse — Bascilian · 2026-11-13