Evaluating Communication Overhead in EP
cHHillee · x · 2026-07-18
This post discusses whether a2a communication costs in EP (expert parallel) are underestimated.
- The author cites Andrew Gu's perspective: in EP, the relative timing of communication and matrix multiplication must be evaluated together.
- The formula provided states the ratio of communication time to matmul time depends on commtimeperbyte / (intermediatedim timeperflop).
- Conclusion: latentmoe only halves inputdim while keeping intermediatedim unchanged, so it doesn't fundamentally change whether the system is network-bottlenecked.
More from Research
- Linear Digressions returns with a new season of audio essays on AI agents — ChrisGPotts · 2026-07-21
- ARISE study tested 45 AI clinical tools in 1,100 consult cases — HealthcareAIGuy · 2026-07-21
- Async OPD distillation doubles throughput while matching synchronous math accuracy — _lewtun · 2026-07-21
- A forecasting lesson on why R-squared alone led to overfitting and worse predictions — mdancho84 · 2026-07-21
- Google DeepMind’s Project Genie talk shows how creatives feed into model research — alexanderchen · 2026-07-21
- Nat Lambert says RL distillation does not use the strongest models as teachers — natolambert · 2026-07-21