Debating AFD Disaggregation and Data Locality in Inference Architecture
GavinSBaker · x · 2026-08-26
In a technical debate, Gavin Baker discussed the compatibility of Attention-FFN disaggregation (AFD) with data locality, specifically the constraint of not moving the KV cache. He argued that OpenAI's Jalapeño project did not make a bet against disaggregation, but rather focused on a topology compatible with data locality: computing prefill and attention on Jalapeño while offloading FFN to another chip. He noted that OpenAI is obviously doing relatively crude disaggregated inference (PD) at scale today. AFD is considered a hard networking problem that offers potential superiority over simpler PD disaggregation approaches.
Related event: KV Cache Offloading Remains Unsolved for Large-Context Agents(2 posts)→
More from Infra
- MiniCPM-V 4.7 runs a 35B MoE multimodal model on phones with just 3GB memory — nevrekaraishwa2 · 2026-10-15
- Nvidia Is Now Worth More Than the Entire Indian Stock Market — theteknosaur · 2026-10-15
- NVIDIA joins EU's AI in Science Enablers, offering compute, training and cloud resources — nordicinst · 2026-10-15
- Open source's hardware trap: data centers crowd out consumer GPUs, local LLMs become a privilege — Dante_77A · 2026-10-15
- NVIDIA backs EU AI in Science push as 35 supercomputers to serve 3M researchers — NVIDIA Blog · 2026-10-15
- Jensen Huang: 2027 AI bottleneck is 35GW of power, revenue justifies the capex — BG2 Pod · 2026-10-15