ARCHead: New LLM Output Head Quantization Method Substantially Reduces Storage with Minimal Loss
Şuayp Talha Kocabay · hf · 2026-08-06
While weight quantization significantly reduces the storage of Transformer blocks, the final language modeling head (LM-head) of LLMs often retains high precision, as naive quantization strongly perturbs the vocabulary logit distribution.
To address this, researchers introduce ARCHead, a compressor combining a quantized low-rank core, group-wise INT4 residuals, and a low-rank correction fitted in an activation-derived metric. This method stores no dense BF16 head and reduces persistent LM-head storage by 3.7-3.9x.
Experiments on Qwen3-8B-Base show ARCHead uses only 25.6% of BF16 head storage while achieving a 1.007 relative perplexity, vastly outperforming storage-matched naive INT4 (1.14-1.16). It serves as a lossless complement to existing block quantizers like AWQ.
More from Infra
- Running DeepSeek V4 Locally on Spark Hardware Hits ~95 tok/s — Rasmic · 2026-08-06
- Local Deployment: Running an NVIDIA and AMD GPU Together for Different Models — Curious-Pen5547 · 2026-08-06
- Luminal Compiler Discovers Insanely Fast Megakernels Without Quantization — AccBalanced · 2026-08-06
- Best Local LLMs for Every Mac: Run a 27B Model on Just 16GB RAM — JosephJacks_ · 2026-08-06
- Running 276B MoE Models on <10GB RAM: Mference Hits ~3 tok/s — Blahblahblakha · 2026-08-06
- RTX 3090 Test: INT8 Quantization Doubles MiniMax Video Generation Speed — Nevaditew · 2026-08-06