PR lands to integrate NF12 format into vLLM with bit-identical outputs
AlpinDale · x · 2026-11-07
A PR to integrate the NF12 format into vLLM is up. After decoding, NF12 weights are identical to the original BF16 model, and with the same execution pipeline they produce exactly the same outputs. Since NF12 moves fewer bytes overall, it yields a speedup roughly proportional to the memory saved — a lossless-bandwidth win for inference.
Related event: Developer open-sources nf12: lossless 12-bit compression for BF16 models(10 posts)→
More from Infra
- HyperFlow H3 Flash: 13.7B Few-Step Model Generates Video With Stereo Audio, Runs on RTX 5090 — Apprehensive_Sky892 · 2026-11-07
- JITEngine synthesizes custom LLM inference engines for Apple Silicon, claims up to 2.7x speedup — AccBalanced · 2026-11-07
- Dev predicts 2027's local inference boom: integrated hardware + OS + inference stack, not just GPUs — vincent_koc · 2026-11-07
- vLLM Office Hours Hits 59 Sessions: 76 Speakers from 24 Organizations and Counting — vllm_project · 2026-11-07
- Dev runs 284B-parameter DeepSeek-V4-Flash on CPU only, squeezed into 64GB DDR5 RAM — solyarisoftware · 2026-11-07
- Omnigraph: open-source graph database with git-style branching for multi-agent shared memory — solyarisoftware · 2026-11-07