Meituan LongCat's QuantMLA: joint INT4 MLA KV caching with 3.59x compression at 128K
meituan-longcat · hf · 2026-10-12
QuantMLA from HKU and the Meituan LongCat team is a function-aligned dual-path quantization framework for MLA KV caching. It models content- and RoPE-path quantization errors (explaining RoPE error amplification), learns offline-fusable path-specific transformations with theoretical distortion bounds, and reports the first joint INT4 content+RoPE caching with minimal accuracy loss. A native low-bit MLA kernel achieves 3.59x cache compression at 128K context and 5.168x higher whole-job throughput than BF16 under cache-pressure serving. INT2 content caching with INT4 RoPE keys remains competitive. Code to be released upon acceptance.
More from Infra
- UK Minister Backs Hardware-Enabled AI Verification, Betting on Britain's Chip Design Expertise — S_OhEigeartaigh · 2026-10-12
- GPU rentals up 27% in six months while fixed-intelligence prices fell 63% — jon_durbin · 2026-10-12
- Broadcom's Hock Tan is betting the company on Anthropic and OpenAI succeeding — firstadopter · 2026-10-12
- AI providers processed 39 quadrillion inference tokens in September, up 7.9x YoY — SuB8u · 2026-10-12
- Cheap drones could torch a $50B data center, researcher urges anti-drone defense — Dr_Singularity · 2026-10-12
- OpenEnv Arena offers free GPU compute to train your environment, weights open sourced — ben_burtenshaw · 2026-10-12