Prime Super Flash MoE: 1.2x BF16 and 1.6x MXFP8 speedups over upstream on B200
retr0jirachi · x · 2026-09-05
After weeks of work on Prime Intellect's Prime Flash MoE, a developer shipped Prime Super Flash MoE, pushing closer to the hardware roofline.
- 1.2x faster on BF16 and 1.6x on MXFP8 vs upstream across 4K–128K tokens on B200
- Fused MoE forward kernels written against NVIDIA Blackwell's tcgen05: gate/up projection, SwiGLU activation, down projection, and top-k weighted reduction are all fused, so the 2H-wide gate/up tensor is never materialized — consumed straight from the accumulator
- Two pipelines via a split flag: split=False runs the whole FFN in one kernel launch with the intermediate activation never leaving the SM; the fork adds persistent BF16/MXFP8 workers, a parallel routing-alignment path, and faster epilogues
Code and an optimization report (methodology, accepted/rejected experiments, machine-readable evidence) are open-sourced on GitHub.
More from Infra
- AMD, Cisco and Saudi Arabia's HUMAIN deploy MI335X GPUs, planning up to 250MW — Beth_Kindig · 2026-09-05
- Reef launches inference-native infra that serves self-improving agents without downtime — pliang279 · 2026-09-05
- From 1D to 2D int8 kernels: a hands-on GPU internals learning path — goyal__pramod · 2026-09-05
- Google DeepMind Publishes Free Book on Scaling LLMs Across TPUs and GPUs — goyal__pramod · 2026-09-05
- How to Estimate tokens/sec on Your Hardware: The VRAM Bandwidth Formula — Pyrolistical · 2026-09-05
- Google Cloud proposes Cloud Run sandboxes to fix coding agents' repair loop — rseroter · 2026-09-05