PN-Mix: 2.26-bit mixed-precision key cache quantization beats 2.32-2.39-bit baselines
pmttyji · reddit · 2026-10-23
A new arXiv paper (2610.19605) proposes PN-Mix, a mixed-precision key-cache quantization method for RoPE-based decoder-only LLMs. Key insight: quantization errors in channel pairs with large query-key pair-norm products drive most attention logit variation—prior methods estimated channel saliency from key statistics alone. PN-Mix identifies salient channel pairs once during prefill and reuses them through decoding. At 2.26-bit average precision it matches or beats prior mixed-precision methods using 2.32-2.39 bits across reasoning, code-generation and AIME benchmarks (best non-FP16 numbers on several), and degrades gracefully down to 1.5 bits where baselines collapse below 2.25 bits.
More from Infra
- Meta researcher: horrifying infra, not talent, is why OSS can't catch closed frontier — A_K_Nain · 2026-10-23
- Jane Street and NVIDIA speed up Mamba-2 with custom cuTile kernels — blelbach · 2026-10-23
- LatentLoop: asymmetric compute allocation cuts looped-MoE training cost by 19% — pmttyji · 2026-10-23
- Same AI request emits 8.5x more CO2 in Texas than in Sweden — jlundberg · 2026-10-23
- SK Hynix Pulls Plug on Solidigm US IPO as Chip Stocks Slide, Shares Down ~30% This Quarter — rohanpaul_ai · 2026-10-23
- Designer mocks Nvidia: buy idle GPUs or lose access to new ones—definitely not a bubble — michalmalewicz · 2026-10-23