BitDecoding-FP4: native FP4 KV cache decoding via algorithm-system co-design
UoE-Mlsys · hf · 2026-11-18
UoE-Mlsys released BitDecoding-FP4, which co-designs FP4 quantization with system optimizations to enable accurate, high-throughput long-context KV cache decoding for LLM inference.
More from Infra
- Lisa Su: Chip demand looks 'generational', not cyclical this time — BenBajarin · 2026-11-18
- Qdrant Generates 32.8M Embeddings in 35 Minutes With Static Embedding Models — qdrant_engine · 2026-11-18
- ComfyUI makes the MiniMax H3 video VAE up to 2.2x faster on Nvidia GPUs — NerdyRodent · 2026-11-17
- ArisGlobal cut Bedrock inference cost bloat by caching shared prompt prefixes first — AWS ML Blog · 2026-11-17
- Cloudflare researcher: Clef's fast release was 'asymmetric effort,' reasoning models next — michellechen · 2026-11-17
- Internet meltdown day 4: brokers hit as Charles Schwab outage blocks traders — GaryMarcus · 2026-11-17