CUDA guide details fine-grained sync: sentinel-based tile readiness over membar, faster on Rubin
simran_s_arora · x · 2026-11-17
- NVIDIA's CUDA Programming Guide (v13.5) adds a fine-grained synchronization chapter: consumer kernels detect when each input tile is ready by inspecting the data itself via a reserved sentinel bit pattern, instead of waiting on flags or producer-grid completion — cutting handoff latency in chains of short, dependent kernels.
- On compute capability 10.7+ devices, this uses TMA data validity reporting: the bulk-async copy checks for the sentinel during global-to-shared transfer and reports via a layout::v1 shared memory barrier that separates copy completion from data readiness. If a sentinel is found, the consumer must reissue the copy; there is no automatic hardware retry.
- farazdotai adds practical advice: on Rubin, further improve fine-grained CTA overlap by using data as the signal (no membar) plus a modified TMA instruction with a hardware data-validity check — doable on Blackwell too, but the hardware check beats software polling.
More from coding & agent
- Open-source Hermes Agent profile: cheap main model plus low-cost side-task pinning — alexcovo_eth · 2026-11-17
- Why Lean? Most momentum among dependent type languages, and agents write it decently — hargup13 · 2026-11-17
- Quarterdeck: open-source Claude Code plugin collapses Anthropic's always-on agent stack into one prompt — InternationalData569 · 2026-11-17
- Dev open-sources a local 'room' where coding agents must get human browser approval for risky commands — Significant_Camp4148 · 2026-11-17
- ComfyUI filmmaker solves character consistency with LoRAs, stuck on background — Interesting_Coyote_4 · 2026-11-17
- Amanda Launches API That Sends AI Agents to Meetings for You — Scobleizer · 2026-11-17