Best Quantization for Sub-2-bit? Beyond QTIP, What Papers to Read?
Aggravating-Push-207 · reddit · 2026-08-12
A user planning to implement their own inference engine for very large LLMs (e.g., Qwen 3.6/8) with only 8GB VRAM is researching sub-2-bit quantization. They found QTIP as the best so far but seek the latest research and paper recommendations.
More from Infra
- Server CPU Market to Reach $220B by 2030, Potentially Split in Thirds — BenBajarin · 2026-08-12
- Hugging Face Model Downloads Drop ~30%, Suspected Platform Filtering Change — natolambert · 2026-08-12
- HuggingFace Models See ~30% Sustained Drop in Daily Downloads, Likely Due to Filtering Changes — natolambert · 2026-08-12
- Michael Burry Slams Nvidia's $500M Chip Financing as Wall Street's Dodgy Debt Scheme — GaryMarcus · 2026-08-12
- Wintermute to Invest $1 Billion in AI Data Center Infrastructure — abhiadesai · 2026-08-12
- 71% of Americans Oppose Nearby Data Centers, Surpassing Nuclear Plants — PeterDiamandis · 2026-08-12