Quantizing the lm_head to 4-bit: self-distillation recovers accuracy, NVFP4 within 0.7 pts
AIFlow_ML · x · 2026-10-22
An analysis of Recursive's new model: with only 2.5B active params, the lmhead accounts for a large share of bytes read per token, so the team put it in 4-bit "against common practice."
The key trick for recovering accuracy: self-distillation from the 16-bit model, and the resulting NVFP4 version lands within 0.7 points of the original. Sources: Recursive's launch post and model card.
Related event: Eureka Machine: 40B MoE activates just 2.5B, with 4-bit lm_head(3 posts)→
More from Infra
- Deutsche Bank Deploys Farsight On-Prem as Financial AI Question Shifts to Where Models Run — huggingface · 2026-10-22
- Gradio distills FLUX.3 Klein's 9B prompt upsampler into 0.8B and 2B models that run on CPU — Gradio · 2026-10-22
- Securing coding agents with Docker Sandboxes and edge LLM inference at ATO — HowDevelop · 2026-10-22
- Revolut's massive outage blamed on GCP, but ex-sales contact says it rejected multi-zone redundancy — thedealdirector · 2026-10-22
- 0.9B Model Transcribes a 4-Person Meeting on a MacBook With 0.08% Speaker Confusion — Binary_orchid · 2026-10-22
- Qdrant 1.20 ships BM25 hybrid search without sparse vectors — qdrant_engine · 2026-10-22