How GPUs organize work: kernel launches, thread blocks and scheduling explained
blaizedsouza · x · 2026-11-22
Avi Chawla publishes a clear visual explainer of GPU work organization, a popular AI engineering interview topic: a kernel launch can create millions of threads that don't all run at once. Threads are grouped into blocks, the GPU assigns groups to chip sections, and within each section instructions from ready threads are scheduled. A linked follow-up article explains why LLM decode waits on memory bandwidth and how a dataflow chip removes that wait, claiming 4x faster inference than GPUs.
More from Infra
- An AI bubble fable: from free cash flow to private credit to annuity blowup — Svwang1 · 2026-11-22
- One RX 7900 XTX runs 524k context at 767 tok/s, 3x faster than llama.cpp — W61k3r · 2026-11-22
- Alibaba's T-Head files for HK IPO at up to $80B, rivaling Baidu's Kunlunxin — 创业邦 · 2026-11-22
- Can llama.cpp RPC span a Mac mini, Mac Studio and dual RTX 3090s to run a 120B model? — ruffus_or · 2026-11-22
- OpenAI's GPT rewrites AMD flagship GPU RTL, gaining up to 14% PPA in 5 months — 新智元 · 2026-11-22
- LBNL report: data center load growth did not drive up residential electricity prices (2019-2025) — Eliv_nurotic · 2026-11-22