Modal engineer: inference consensus converging on disagg + wide EP + KV storage
dejavucoder · x · 2026-11-11
Modal engineer ekzhang1's day-2 notes: the company has scaled but kept its soul, and the team is exceptionally strong. He reports that in this week's inference debates, opinion has visibly converged on disaggregated serving + wide expert parallelism + KV storage, with platform work coming next. dejavucoder joked about wanting daily updates like this.
More from Infra
- Huawei's TSMC 'White Glove' Chip Orders Resurface in Debate over Megaspeed's $4.8B Nvidia Buys — 0xkarasy · 2026-11-11
- Data Center Backlash Spreads: Community Outrage Kills Upstate NY Project on Abandoned Steel Plant — sebkrier · 2026-11-11
- AMD's AI World Tour Japan: global monthly token consumption up 158x in two years — DavidBennett__ · 2026-11-11
- vLLM crosses 4,000 GitHub contributors, growing from a Berkeley research project — vllm_project · 2026-11-11
- 40k agent calls show prefix stability, not model choice, drives 4-5x cost gaps — MichelleZhao91 · 2026-11-11
- LMCache taps Phoenix GPU direct storage, cutting TTFT 5.6x on NVMe-backed KV caches — AccBalanced · 2026-11-11