Hardware-aware LDM implementation cuts memory traffic via state unwinding and windowed writes
SonglinYang4 · x · 2026-11-09
Songlin Yang endorses Korbi Poeppel's pointer to a hardware-aware implementation (Sec 3.3) of LDM, which reduces both activation memory overhead and memory traffic for layer-wise state reads/writes (RLS recurrence):
- State unwinding: The RLS recurrence is reversible, so the backward pass reconstructs states from compact per-layer auxiliary context instead of storing the full G×Dv×Dk state per layer.
- Windowed writes: Writes are fused with the next layer's reads and deferred over a window of E=4 layers, avoiding materializing intermediate states and their gradients.
- Implemented in CuTe DSL for Hopper and Blackwell GPUs.
More from Infra
- AI chip design demo: one prompt for a 64-bit RISC-V processor under 3W TDP — satnam6502 · 2026-11-09
- Appwrite launches MCP1: one URL for every MCP server your team uses — eldadfux · 2026-11-09
- Saudi's HUMAIN and National Infrastructure Fund line up up to SAR 2.5B for AI infrastructure — HUMAIN · 2026-11-09
- Local models' broken chat templates silently broke agent clients — here's the fix — viest_me · 2026-11-09
- 48% speedup: optimized llama.cpp settings for Qwen3.8-Flash-Next on Ryzen AI Max+ 395 — Prudence-0 · 2026-11-09
- Day 309/365 of GPU Programming: Optimizing LLaMA2-7B on Edge with SIMD and Multithreading — NandoDF · 2026-11-09