BF16 is near-lossless already; efficient kernels are the real barrier for lower bit-widths
AlpinDale · x · 2026-11-07
Asked whether a quantization method generalizes to FP32, @AlpinDale replies: possible in theory but pointless — BF16 is already so close it might as well be lossless. The method could theoretically apply to any bit-width, but mathematical issues make writing efficient inference kernels for most of them a nightmare.
Related event: Developer open-sources nf12: lossless 12-bit compression for BF16 models(10 posts)→
More from Infra
- US AI scale now hinges on coordinating power, components, agentic workflows and humanoid pilots — ingliguori · 2026-11-07
- Fudan's D3D diffusion drafter enables async speculative decoding, up to 7.48x speedup on Qwen3 — pmttyji · 2026-11-07
- Study: data centers have only marginal impact on residential electricity prices — TinfoilTricorn · 2026-11-07
- YouJiacheng: post-training needs high tps for long-tail rollouts — YouJiacheng · 2026-11-07
- vLLM contributor: lossless KV cache quantization viable but not worth it — AlpinDale · 2026-11-07
- NVIDIA's first co-packaged optics Ethernet switch, Spectrum-X Photonics, enters full production — TinfoilTricorn · 2026-11-07