Sliding Window Attention Explained with Cartoon Huangs: Old Tokens Slide Out of View
cfregly · x · 2026-11-11
cfregly released excerpts from his new "AI Systems Performance Engineering, the Huang Edition" series, using cartoon characters Jensen, Lori, and Alice Huang to illustrate sliding window attention (SWA): each query sees only its own slot plus the 2 previous ones, so older tokens like Lori drop out of view — a "selective memory" that's usually a feature. The post links the Mistral 7B paper, where SWA handles arbitrary-length sequences at reduced inference cost.
Related event: Cartoon Series Explains Sliding Window Attention and Top-k Sampling(2 posts)→
More from Infra
- Agent Arena: Gemini 4 Carbon Gets More Cost-Competitive as Tasks Get Harder — rseroter · 2026-11-11
- AMD Zen 6 deep dive: cheaper branch mispredicts and a return to Zen 3/4 integer scheduling — jedisct1 · 2026-11-11
- Together AI SVP Dan Fu on kernels, serving 3T-param models day zero, and self-improving inference — togethercompute · 2026-11-11
- ExecuTorch adds WebGPU backend: run PyTorch models in-browser, up to 10x faster than CPU — PyTorch · 2026-11-11
- Hedra runs training experiments with AI agents steered by human researchers — testingcatalog · 2026-11-11
- Simular Box gives Claude, ChatGPT and Cursor agents their own cloud Linux desktop via MCP — AdFluid9823 · 2026-11-11