Two weeks of quant testing on a 5080: above Q4, most quants were statistically indistinguishable
KitchenAmoeba4438 · reddit · 2026-08-23
The author ran two straight weeks of quantization comparisons on an RTX 5080 (16GB) and concluded the opposite of community consensus: quantization isn't a continuous "quality dial" — most of the dial isn't connected.
Key findings:
- Damage starts below Q4; most quants above it were statistically indistinguishable from each other under these tests
- QAT breaks on mismatch: a model QAT-trained at one quant run at another performs badly — match the trained quant or pick a different model
- MoEs are less affected than dense models
- Gaps between common quants were smaller than consensus predicts; the author went in expecting to confirm consensus and the data didn't support it
Sessions were deliberately 2-4 messages, testing whether a quantized model is intact, not how it holds up under load. The next round targets DevOps, coding, and long sessions, where the author expects a VRAM/speed-vs-accuracy tradeoff rather than a simple ranking. Raw data, benchmark code, and datasets are open-sourced in the article's git repo.
Related event: Quantization below Q4 noticeably degrades local Agent models(3 posts)→
More from Infra
- Copper to optical: optical backplanes seen as necessary as racks move to optics — BenBajarin · 2026-10-16
- Polymarket prices 28% odds a US state enacts data center moratorium by year-end — Polymarket · 2026-10-16
- Pennsylvania residents reject $10,000-per-household offer for massive Amazon data center — Polymarket · 2026-10-16
- PyTorch now runs natively on AWS Trainium with zero code changes — PyTorch · 2026-10-16
- Chinese labs run on ~1/10 US compute; ByteDance pays 2-4x to hoover up SEA supply — robleclerc · 2026-10-16
- Zymtrace raises $12.2M and hires ML engineers to build agents that fix GPU bottlenecks — cfregly · 2026-10-16