Prime Flash MoE: Blackwell-Optimized CUDA Kernels Speed Up MoE Inference by 2.4x
pbaylies · x · 2026-08-14
Prime Intellect releases Prime Flash MoE, a set of Blackwell-optimized CUDA kernels for MoE inference. By fusing routing-aware GEMMs, SwiGLU, quantization, and reductions, it avoids materializing intermediate tensors in HBM, saving memory traffic. Up to 2.4x faster than PyTorch grouped GEMM, with 2.3x speedup across 4k-128k tokens. Supports BF16 and MXFP8, with fused and split pipelines. Code is open-sourced.
Related event: Prime Intellect Releases Prime Flash MoE to Optimize Blackwell Inference(3 posts)→
More from Infra
- Drama3 TTS lands in audio.cpp: up to 5x real-time on RTX 5090, <8GB VRAM mode too — Acceptable-Cycle4645 · 2026-10-16
- Three CoreWeave co-founders have sold $3.3B in shares since IPO amid credit worries — SumitGup · 2026-10-16
- Cherry pi on a Raspberry Pi: the tiny-model rumor developers are watching — sull · 2026-10-16
- CoreWeave's debt costs falling: DDTL spreads tightened from S+9.62% to S+2.25% — BenBajarin · 2026-10-16
- SoftBank lines up US IPOs for SB Energy and AI robotics spinoff Roze, valuations top $50B — nordicinst · 2026-10-16
- TRL reproduces Modal's mixed-precision async RL: plain GRPO collapses, SC+MIS holds 0.72 — QGallouedec · 2026-10-16