RTX 5090 local tests show Qwen Q6 can drop to 15 tok/s at 80k context

LFAdvice7984 · reddit · 2026-07-27

Local Qwen quantization tests on a 5090 show big speed differences between Q5, Q6, and CPU-only setups.

• The author is trying to summarize a very large document with an 80k context window.
• Q6 quantization of Qwen3.6 27B appears to hit VRAM limits and only reaches about 15 tok/s, even with KV cache quantized to q4 and concurrency set to 1.
• A Q5 quant fits comfortably and runs at roughly 60–70 tok/s.
• Qwen35 A3B reaches about 90–100 tok/s.
• The author suspects the setup is misconfigured, since another user reported 130k context and 130 tok/s with Q6.
• Hardware: 64 GB system RAM and a Ryzen 7950X; the test is on Windows using LM Studio.

Original post →

More from Infra

• Nvidia gets mocked as “the leading open-source AI company” while repo chart shows it ahead — AccBalanced · 2026-07-27
• $8 ESP32-S3 runs a 28.9M-parameter LLM fully offline at 9.5 tokens per second — yangyi · 2026-07-27
• YC talk on BCI x AI says infrastructure is what really determines speed — garrytan · 2026-07-27
• A 13B model ran on a no-GPU PC by paging weights from SSD via llama.cpp — ID_R_McGregor · 2026-07-27
• llama.cpp warns that GGUFs made before a recent change must be regenerated — EconomySerious · 2026-07-27
• Surprising Ubuntu Setup: NVIDIA 5090 PC Becomes the Easiest AI Rig — _xjdr · 2026-07-27

Infra channel →