NVIDIA BioNeMo Speeds Up MoE Training Up to 2.21x Over HF Baseline

NVIDIA's BioNeMo team optimized MoE training for biological foundation models, using techniques like GroupedLinear to merge expert GEMMs, achieving up to 2.21x training throughput over the Hugging Face BF16 baseline for Mixtral-8x7B on 8 B200 GPUs.

2026-09-29 ~ 2026-10-01 · 3 related posts