CroissAttn runs all six attention matmuls in FP4 on Blackwell: 1.5× throughput, 55% less memory
TristanThrush · x · 2026-11-07
CroissAttn demonstrates that all six attention matmuls can run in FP4 on NVIDIA Blackwell. At 128K context, Qwen3-8B trained with it achieved 1.5× throughput and 55% less GPU memory vs BF16. Reposter Anupama Dhar adds that training fully or fine-tuning in FP4 also improves downstream accuracy of FP4 serving — end-to-end low precision without sacrificing accuracy. Technical details to be presented at Nebius Paper Club.
More from Infra
- Reactor co-founder publishes 'The real-time manifesto' on serving real-time AI at scale — smtabatabaie · 2026-11-07
- Clairvoyance's new hivemind feature lets you share your GPU with anyone — draginol · 2026-11-07
- zml/llmd inference server runs on NVIDIA, AMD, Intel, TPU and Metal with images as small as 142MB — siwu · 2026-11-07
- Sequoia's Maguire models SpaceX taking inference compute share as neoclouds struggle — shaunmmaguire · 2026-11-07
- Musk teases 'Starmind computer' details next year, promising it will 'blow people's minds' — ns123abc · 2026-11-07
- Airbnb fine-tuned Gemma 4 E2B to match frontier quality at 1% of the cost — DynamicWebPaige · 2026-11-07