NF12: a lossless BF16 weight format that drops redundant exponent bits for 25% savings
AlpinDale · x · 2026-11-07
AlpinDale explains the NF12 weight encoding: over 99.7% of trained weights share the same upper four exponent bits, so groups of 8 weights pack into 12 bytes instead of 16. Weights that can't be represented are stored exactly in escape buffers (28 bytes per escaped group), making decoding bit-identical to BF16 while giving a speedup roughly proportional to the memory saved. Escape overhead only becomes a problem below NF12 (e.g. NF11/10).
Related event: Developer open-sources nf12: lossless 12-bit compression for BF16 models(10 posts)→
More from Infra
- YouJiacheng: post-training needs high tps for long-tail rollouts — YouJiacheng · 2026-11-07
- vLLM contributor: lossless KV cache quantization viable but not worth it — AlpinDale · 2026-11-07
- NVIDIA's first co-packaged optics Ethernet switch, Spectrum-X Photonics, enters full production — TinfoilTricorn · 2026-11-07
- This week's open training stack: MoE training handbook, harness adapters, and a harness leaderboard — ariG23498 · 2026-11-07
- Musk: space will be the cheapest place to run superintelligence within 2-3 years — TinfoilTricorn · 2026-11-07
- JITEngine synthesizes custom LLM inference engines for Apple Silicon, claims up to 2.7x speedup — AccBalanced · 2026-11-07