EchoMoE Sends Only Deltas Between Nodes, Cutting MoE Inference Traffic 58%
sivareddyg · x · 2026-11-19
EchoMoE tackles the shift of MoE inference from compute-bound to communication-bound: instead of shipping full activations between GPUs hosting different experts, it transmits only the deltas. The retrofitting technique works on pretrained MoEs (no from-scratch training like LatentMoE), cutting inter-node traffic by 58% and boosting throughput by 22.6%.
More from Infra
- JEDEC unveils DDR6 spec: 12.8 GT/s起步、最高 14.4 GT/s,带宽提升 50%,支持 128Gb 颗粒 — gnukeith · 2026-11-19
- Datadog and Baseten fine-tune Qwen 3.5 9B for a 50x faster LLM summarizer — baseten · 2026-11-19
- Nvidia posts $116B quarter as analyst moves rating toward risk over unresolved uncertainties — BenBajarin · 2026-11-19
- Oracle co-CEO admits 'classic mistake' selling data centers to towns — jeremyakahn · 2026-11-19
- Surface Laptop Ultra With NVIDIA RTX Spark Runs 94GB Model, Finishes 4+ Enterprise Agent Workflows Per Hour — ryanshrout · 2026-11-19
- Nace ships 71.8GB DeepSeek V4.1 Flash coding quant with 39% lower KL than rivals — m4rkGS · 2026-11-19