Prime Flash MoE: Blackwell-Optimized CUDA Kernels for MoE Inference
sloppenheimer · x · 2026-08-14
PrimeIntellect introduced Prime Flash MoE, a set of Blackwell-optimized CUDA kernels designed for Mixture-of-Experts (MoE) inference.
Core Optimizations & Performance:
- Avoids materializing intermediate tensors and activations (in fused mode), significantly saving memory traffic.
- Delivers up to 2.4× speedup over PyTorch grouped GEMM and 2.3× speedup across the 4k-128k token range.
- Integrated into their prime-rl framework to accelerate the forward pass of MoE models.
Dual Data Paths:
- Features paths for both BF16 and MXFP8, sharing the same structure but differing in how they feed tensor cores.
- Offers fused and split pipelines. Fusion avoids activation round-trips for small problem sizes, while the split pipeline becomes more cost-effective as the working set grows by materializing smaller activations.
Related event: Prime Intellect Releases Prime Flash MoE to Optimize Blackwell Inference(3 posts)→
More from Infra
- New hardware day: local LLM deployment the moment it arrives — lxfater · 2026-10-16
- Together AI's score centering fixes RL training collapse under quantized inference — nrehiew_ · 2026-10-16
- Inference is splitting prefill and decode across specialized chips, not just H100s — AccBalanced · 2026-10-16
- Rebasely launches in beta: Git hosting built for humans and coding agents — handotdev · 2026-10-16
- Drama3 TTS lands in audio.cpp: up to 5x real-time on RTX 5090, <8GB VRAM mode too — Acceptable-Cycle4645 · 2026-10-16
- Three CoreWeave co-founders have sold $3.3B in shares since IPO amid credit worries — SumitGup · 2026-10-16