PACE: chunked 4-bit recurrent-state quantization matches 8-bit at 2/3 memory
pmttyji · reddit · 2026-10-23
A new paper introduces PACE, a post-hoc 4-bit quantizer for linear-attention language model recurrent states, enabling more concurrent decode sequences per accelerator.
- Rounds the state only once every few tokens (buffering in-between tokens) instead of per-token, reducing accumulated error.
- Exploits that large gated-delta state entries cluster in columns, sharing quantization scales per column; heads with longest error retention stay at 8 bits.
- On four hybrid models (4B–35B), mixed 4/8-bit PACE matches 8-bit DAMP on the three larger models at 2/3 memory; full 4-bit beats Q-Mamba at equal memory while preserving decode speedup.
arXiv: 2610.29545
Related event: PACE: 4-bit state quantization for linear attention nears 8-bit accuracy(2 posts)→
More from Infra
- TokenSpeed open-source inference engine hits 2.8k stars with Day 0 support and spec decoding — zhyncs42 · 2026-10-23
- Europe can't build one giant AI datacentre — distributed training across sites may fill the gap — gklambauer · 2026-10-23
- ReaDy Speculative Decoding Hits 3.16x Speedup on Qwen3.8-27B With a Single B200 — pmttyji · 2026-10-23
- 3 Lessons from Rebuilding Compute for AI Agents: Safe Sandboxes, Near-Free Idle, Persistent Memory — Saboo_Shubham_ · 2026-10-23
- Intel CTO confirms partnership with Musk on Terafab: 'We either build it or we don't have the chips' — elonmusk · 2026-10-23
- Researcher: Putting a $200K GPU Server Purchase Order in a Paper Envelope — HildeKuehne · 2026-10-23