FLAMIA: MXFP8 microscaling attention kernel for fast, causally-safe training on Blackwell
yresearch · hf · 2026-11-03
FLAMIA is a Flash Attention implementation for NVIDIA Blackwell GPUs that accelerates Transformer attention training using MXFP8 microscaling and mixed-precision kernels.
- Shares scale factors per key-group to cut quantization overhead while remaining causally safe — preventing information from future tokens leaking via shared scales past each position's causal boundary;
- Supports variable-length packed documents, using a dense per-row scale table and 128×4 atoms in shared memory to zero cross-document contributions within packed rows.
More from Infra
- KVCache Meetup Korea dives into LLM serving: CXL pooling, roofline analysis, batch scheduling — PyTorch · 2026-11-03
- FuriosaAI's new round values it at 662x last year's sales, no chipmaker joined — YvesMulkers · 2026-11-03
- Just upgrading onnxruntime-gpu 1.23→1.31 with CUDA 13 made SAM 3 inference 3.8x faster — MaziyarPanahi · 2026-11-03
- Dev argues buying GPUs for local AI is just an expensive hobby — API and rented GPUs are always cheaper — Angaisb_ · 2026-11-03
- Local inference devs grapple with models breaking their own CoT on streamed tokens — SilentMobius · 2026-11-03
- 2,500+ GW of power projects stuck in grid queues as $200B flows into energy tech — ingliguori · 2026-11-03