Reddit asks: is 2-bit or 3-bit quantization actually usable for daily LLM work?
Queasy_Back2588 · reddit · 2026-10-18
A Reddit user kicks off a practice-oriented discussion on quantization choices for local LLMs, asking for real-world experience over benchmarks: which quant levels people actually run for chat, coding, and agents; whether 2-bit is usable or too degraded; if 3-bit suffices for complex coding work or you need 4-bit; and what KLD score people consider 'basically lossless' — 0.1, or closer to 0.01–0.03. Useful thread to watch for community consensus on local deployment tradeoffs.
More from Infra
- Musk's Terafab vision: AI chip race shifts to full vertical integration for speed — tekbog · 2026-10-18
- Hands-on: Surface Laptop Ultra + NVIDIA RTX Spark tested for local AI — draginol · 2026-10-18
- IBM hands chip design to an open AI agent factory using distributed inference — emmanuelvivier · 2026-10-18
- Nvidia warns customers of 15%+ AI server price hikes amid memory shortage — emmanuelvivier · 2026-10-18
- Microsoft finishes pre-training 2.8T-param MAI-Base-2 on 16,000 GB200 GPUs in 38 days — emmanuelvivier · 2026-10-18
- KAIST and DeepMind propose Declarative Attention: models declare which context they need, cutting KV reads by 46% — coallaoh · 2026-10-18