$1400 decade-old V100 32GB runs Qwen 27B: Q4_K_M +48% gen speed, Q8_0 +16% prefill

PuzzleheadedArm3732 · reddit · 2026-11-22

A Redditor tested Qwen3.8-27B quantization levels on a $1400 decade-old setup (Dell Precision 5820, 64GB DDR4, V100 PCIe 32GB capped at 225W), fully in VRAM with stock llama.cpp: Q4KM generates 48% faster than Q80, while Q80 prefills 16% faster (10% at 32k context); deep context costs 15% generation speed overall (Q4KM slides from 35.5 to 29.5 tok/s at 32k). For reference, Qwen3.5-35B-A3B (MoE) hits 104 tok/s on the same box but is less capable than the 27B dense. Verdict: the V100 remains very usable.

Original post →

More from Infra

Infra channel →