MegaMoE kernel posts 4x per-GPU throughput vs baseline on 8xH200 DeepSeek V4.1 Flash
michellechen · x · 2026-11-01
A lighthearted exchange between michellechen and Kevin points to the real payload in the attached benchmark chart: running DeepSeek V4.1 Flash 1017 on 8x H200, comparing three inference setups — a custom DEP8 MegaMoE kernel, the open-source DEP8 (DeepInfer), and the pre-DeepInfer baseline.
Key numbers:
- At 100 TPS per-request latency (p50), MegaMoE delivers roughly 4x the per-GPU throughput of the baseline (2400 vs 600 total TPS/GPU)
- DEP8 MegaMoE peaks near 4900 total TPS/GPU while keeping strong per-request speed
- Open-source DeepInfer already beats the baseline clearly, but the custom MegaMoE pulls further ahead in the low-to-mid latency range
For teams deploying MoE models, the curves suggest deep expert-parallel optimization can cut serving costs dramatically at interactive latencies.
More from Fun
- Halloween meme: AI-generated pumpkins turn and smile wider than you carved them — LudovicCreator · 2026-11-01
- Parody arXiv paper jokes that GPT-5.6 Sol reproduced an egalitarianism proof unaided — teortaxesTex · 2026-11-01
- Explaining the AI world to normies, Arrival-style: the meme that nails the mood — dejavucoder · 2026-11-01
- BooBench: A Halloween Benchmark That Scares LLMs and OCR Tools Into Character-Repeating Meltdowns — xennygrimmato_ · 2026-11-01
- Local Maximus meme: six RTX 3090s, 256GB RAM, servant to no API — gklambauer · 2026-11-01
- TinyHand: a free open-source tiny hand pointer for your Mac — SuB8u · 2026-11-01