Benchmarked 21 Qwen3.8-27B quants on 16GB VRAM: bartowski IQ4_XS wins
Storterald · reddit · 2026-09-05
A Reddit user benchmarked 21 quantized variants of Qwen3.8-27B on an RTX 5080 (16GB) using their own C code, ranked by Mean KLD (output distribution divergence from the original model).
Key findings:
- Best overall: bartowski/Qwen3.8-27B-IQ4XS (KLD 0.056, 14.5GiB, fits in 16GB)
- Best uncensored: huihui-ai/Huihui-Qwen3.8-27B-abliterated-UD-IQ4XS (KLD 0.083)
- For smaller footprints: jpetrina's IQ4XS-pure or unsloth's UD-Q3KXL (KLD 0.14, just over 12GiB)
- unsloth UD-Q4KXL (KLD 0.028, 16.4GiB) is the quality ceiling but doesn't fit
- 2-bit quants (sdkyuan QAT Q20, ISTA-DASLab GSQ-RCO IQ2 series) degrade significantly
Overall: 4-bit is nearly lossless, 3-bit quite usable, 2-bit noticeably worse.
More from Infra
- PostgreSQL 19 slips to November as LLM-assisted code review floods committers with edge cases — sanjaykalra · 2026-10-21
- Decision Models' routing classifier hits AntSeed, cutting inference cost to 24% of always-Opus — airesearch12 · 2026-10-21
- Eon cuts engineer onboarding from 2 days to 2 minutes with an MCP Gateway on Bedrock AgentCore — AWS ML Blog · 2026-10-20
- a16z's Ben Horowitz says AI buildout isn't the 90s bubble — an investor pushes back — henloitsjoyce · 2026-10-20
- Micron and Nvidia drive ~36% of S&P 500's Q3 earnings growth — Beth_Kindig · 2026-10-20
- NVIDIA lands in Europe: Jensen Huang and German chancellor discuss industrial AI at GTC Berlin — nvidia · 2026-10-20