TensorSharp Runs 816B MoE Model Across 10 A40 GPUs

The author of TensorSharp, an open-source .NET LLM inference engine, ran the Muse Spark 816B MoE model (42B active, 316 GiB weights) on 10 A40 GPUs, finding pipeline-parallel layer splitting makes prefill 4.6x faster than tensor parallelism.

2026-10-23 ~ 2026-10-23 · 2 related posts

• Running an 816B MoE on 10× A40s: layer split beats tensor parallelism 4.6× on prefill — fuzhongkai · 2026-10-23
• Running an 816B MoE across 10 GPUs: pipeline parallelism beats tensor parallelism 4.6x on prefill — fuzhongkai · 2026-10-23