Developer open-sources nf12: lossless 12-bit compression for BF16 models

Developer AlpinDale has open-sourced the nf12 library, which losslessly compresses BF16 model weights into fixed-length 12-bit encodings. The approach is based on Together Compute's NyoomFloat12 paper (authors Liberman, Fang, Zhang, Dao, Fu, 2026) and supports GPUs from Ampere through Blackwell. Notably, the compression is fully lossless: decoded weights are bit-for-bit identical to the original BF16 values, and model outputs are exactly the same under an identical execution pipeline.

Confirmed

• How it works: BF16's exponent range far exceeds what post-training weights actually need. Over 99.7% of weights in a model share the same top four exponent bits (0111); these bits don't need to be stored and are restored at decode time just like traditional file compression, making the encoding lossless.
• Implementation details: Not every weight is strictly 12-bit. Weights are processed in groups of 8: normal groups are packed from 16 bytes down to 12, while groups that can't be represented are flagged as outliers and their original BF16 values written to an escape buffer (28 bytes per escaped group), so total size almost never exceeds BF16.
• Performance: Decoded models are bit-identical to the original BF16 weights; since fewer bytes need to be moved, speedups scale roughly with the memory saved — a bandwidth dividend at zero precision cost.
• Ecosystem progress: AlpinDale has submitted a PR to integrate the NF12 quantization format into vLLM.

Why it matters

Quantization usually trades accuracy for memory and speed, but NF12 offers bit-exact lossless compression, giving users roughly a quarter of memory savings plus corresponding bandwidth gains at no precision cost. If the vLLM integration merges, mainstream inference stacks will be able to load NF12-format models directly — a practical win for LLM deployment costs.

2026-11-07 ~ 2026-11-07 · 10 related posts

• [source] nf12 open-sources lossless 12-bit compression of BF16 models, runs on Ampere through Blackwell — AlpinDale · 2026-11-07
• BF16 models losslessly compressed to 12-bit: 99.7% of weights share exponent bits — AlpinDale · 2026-11-07
• [source] NF12: a lossless BF16 weight format that drops redundant exponent bits for 25% savings — AlpinDale · 2026-11-07
• How NF12 works: escaped groups take 28 bytes, rarely exceeding BF16 size — AlpinDale · 2026-11-07
• NF12 quantization is lossless vs BF16, with speedup proportional to memory saved — AlpinDale · 2026-11-07
• [source] PR lands to integrate NF12 format into vLLM with bit-identical outputs — AlpinDale · 2026-11-07
• Lossless BF16-to-12-bit model compression ships with efficient NyoomFloat12 kernels — MaleManlpulator · 2026-11-07
• Extensive experiments show weight quantization bottoms out around 10.45 bits — AlpinDale · 2026-11-07
• BF16 is near-lossless already; efficient kernels are the real barrier for lower bit-widths — AlpinDale · 2026-11-07
• Why the BF16-to-12-bit compression trick won't extend to FP8 or FP32 weights — AlpinDale · 2026-11-07