Debating AFD Disaggregation and Data Locality in Inference Architecture
GavinSBaker · x · 2026-08-26
In a technical debate, Gavin Baker discussed the compatibility of Attention-FFN disaggregation (AFD) with data locality, specifically the constraint of not moving the KV cache. He argued that OpenAI's Jalapeño project did not make a bet against disaggregation, but rather focused on a topology compatible with data locality: computing prefill and attention on Jalapeño while offloading FFN to another chip. He noted that OpenAI is obviously doing relatively crude disaggregated inference (PD) at scale today. AFD is considered a hard networking problem that offers potential superiority over simpler PD disaggregation approaches.
Related event: KV Cache Offloading Remains Unsolved for Large-Context Agents(2 posts)→
More from Infra
- AI infra spending surges to RMB 11B with 50,000 Hopper chips acquired — zephyr_z9 · 2026-08-27
- Gavin Baker on OpenAI chips: Jalapeño struggles against GPU economics — GavinSBaker · 2026-08-27
- Amazon forced to renewables after building 40 data centers — aronchick · 2026-08-27
- Anthropic reportedly agrees to $45B AI compute rental deal — Polymarket · 2026-08-27
- List of AI labs globally with significant compute resources — BlancheMinerva · 2026-08-27
- How to achieve same intelligence with less computation? Exploring 7 technical paths — prateekj · 2026-08-27