MoE Training Communication Slashed via 8-bit Quantization and Fused Kernels

A community discussion covers MoE training communication optimizations: RHT plus S2F6 quantization compresses combine traffic from 16 to 8.25 bits, NVFP4 dispatch cuts NVLink transfers, and the BBBK kernel fuses quantization into the GEMM epilogue to saturate both NVLink and Tensor Cores.

2026-11-14 ~ 2026-11-14 · 4 related posts

• Quantizing training to MXFP8 with stochastic rounding: the accuracy-speed sweet spot — snowclipsed · 2026-11-14
• Quantized all-to-all: NVFP4 dispatch plus RHT+S2F6 8-bit combine to fix MoE comms bottlenecks — snowclipsed · 2026-11-14
• MoE combine traffic cut from 16 to 8.25 bits per value via RHT + S2F6 quantization — snowclipsed · 2026-11-14
• BBBK kernels fuse quantization into GEMM epilogue, saturating NVLink and Tensor Cores — snowclipsed · 2026-11-14