Prime Intellect makes DeepSeek's MegaMoE trainable, up to 55% faster MoE training
samsja19 · x · 2026-10-30
- Prime Intellect released Prime Mega MoE, making DeepSeek's Mega MoE kernel trainable by fusing computation and communication in the MoE layer and adding backward kernels.
- Benchmarks on two Qwen3 models: up to 55% faster MoE training than TorchTitan, and up to 34% faster hosted training with prime-rl.
- Supports BF16, MXFP8, and FP8 on both Blackwell and Hopper GPUs.
More from Infra
- Ollama says enterprise teams worldwide are moving workflows to open models — ollama · 2026-10-30
- Rogers Expands Starlink Satellite Roaming to Japan, Australia and New Zealand — XFreeze · 2026-10-30
- Corey Quinn flags macOS update silently restoring the removed eMudhra root cert — kingslyj · 2026-10-30
- NVIDIA shares guide on maximizing memory efficiency and cutting costs on Jetson — NVIDIAAI · 2026-10-30
- Rebuild NVIDIA's speaker-captions demo with one transformers serve command — yb2698 · 2026-10-30
- Cap moves transcription to its own GPU farm, up to 11x faster — JosephJacks_ · 2026-10-30