NF12 quantization is lossless vs BF16, with speedup proportional to memory saved
AlpinDale · x · 2026-11-07
Details on the NF12 quantization format: after decoding, the model is bit-identical to the original BF16 weights, so the same execution pipeline produces exactly the same outputs. Since NF12 moves fewer bytes overall, it gets a speedup roughly proportional to the memory it saves.
Related event: Developer open-sources nf12: lossless 12-bit compression for BF16 models(10 posts)→
More from Infra
- YouJiacheng: post-training needs high tps for long-tail rollouts — YouJiacheng · 2026-11-07
- vLLM contributor: lossless KV cache quantization viable but not worth it — AlpinDale · 2026-11-07
- NVIDIA's first co-packaged optics Ethernet switch, Spectrum-X Photonics, enters full production — TinfoilTricorn · 2026-11-07
- This week's open training stack: MoE training handbook, harness adapters, and a harness leaderboard — ariG23498 · 2026-11-07
- Musk: space will be the cheapest place to run superintelligence within 2-3 years — TinfoilTricorn · 2026-11-07
- JITEngine synthesizes custom LLM inference engines for Apple Silicon, claims up to 2.7x speedup — AccBalanced · 2026-11-07