BitDecoding-FP4: native FP4 KV cache decoding via algorithm-system co-design

kalyan_kpl · x · 2026-11-20

A new paper presents BitDecoding-FP4, an algorithm-system co-design enabling LLM decoding with native 4-bit floating-point (FP4) KV caches while preserving BF16-level accuracy.

Problem

Approach

A directly applicable engineering direction for memory/bandwidth bottlenecks in long-context serving.

Original post →

More from Infra

Infra channel →