BitDecoding-FP4: native FP4 KV cache decoding via algorithm-system co-design

UoE-Mlsys · hf · 2026-11-18

UoE-Mlsys released BitDecoding-FP4, which co-designs FP4 quantization with system optimizations to enable accurate, high-throughput long-context KV cache decoding for LLM inference.

Original post →

More from Infra

Infra channel →