PACE: 4-bit state quantization for linear attention nears 8-bit accuracy

Humboldt University's PACE quantizes linear-attention recurrent states to 4-bit post-hoc, rounding only every few tokens to approach 8-bit accuracy while cutting memory without fine-tuning.

2026-10-22 ~ 2026-10-23 · 2 related posts

• PACE: Post-Hoc 4-Bit Quantization of Linear-Attention Recurrent States Without Fine-Tuning — huberlin · 2026-10-22
• PACE: chunked 4-bit recurrent-state quantization matches 8-bit at 2/3 memory — pmttyji · 2026-10-23