Tri Dao on the Next Big Unlock: Kernels, Inference Stacks, and Fleet Optimization
togethercompute · x · 2026-08-13
During an interview at ICML, FlashAttention creator Tri Dao shared his perspective on the evolution of AI architectures. When asked where the next major architectural unlock will come from, he offered a contrarian view: there isn't a single silver-bullet architecture.
Instead, he argues that the real performance unlocks lie in layer-by-layer optimizations—specifically in kernels, inference stacks, and fleet-level optimization. He also emphasized that most of this critical infrastructure work is currently happening in the open-source community.
More from Infra
- Tencent squeezes 770B-parameter Hunyuan Hy4 Preview into 214 GiB at ~2.38 bits per weight — coolmum18 · 2026-10-16
- TokenSpeed MLA 2 hits NVIDIA Vera Rubin: up to 3.44x faster decode vs B200 — zhyncs42 · 2026-10-16
- US vs Chinese hyperscalers: Americans fund and schmooze, Chinese demand your BoM and cap you at 7% margin — jwt0625 · 2026-10-16
- How LiteLLM routes one OpenAI-shaped API across every model provider — Mahmoud_Zalt · 2026-10-16
- OpenAI-Broadcom custom AI chip Jalapeño named a TIME Best Invention of 2026 — itsclivetime · 2026-10-16
- Developer live-runs continual pretraining of a Luganda LLM on Colab, 41% done with loss dropping fast — weskambale · 2026-10-16