Thread coarsening cuts CUDA matmul kernel to 0.41ms on T4, 1.7x speedup with tiling
goyal__pramod · x · 2026-09-04
Developer shares a CUDA matmul kernel optimization journey: the first version averaged 0.598ms on a T4 (BATCH=8, M=N=K=256); basic tiling brought it to 0.348ms (1.7x), and thread coarsening along one dimension pushed it further down to 0.41ms territory. A hands-on look at tiling plus thread coarsening gains.
More from Infra
- Hermes adds local backend: run Unsloth UD-Q4 quants of DeepSeek-V4-Flash and Qwen3.8 one-click — danielhanchen · 2026-09-04
- MLX-Serve v26.9.1 Ships: One-Shot Qwen Flash Runs from a Single Screenshot — TheMoonMidas · 2026-09-04
- Dev says Cloudflare's Wrangler CLI is the most agent-friendly way to run cheap infra — dinasaur_404 · 2026-09-04
- LLM Token Expenditure Index Falls Below $1, Down Over 50% From Summer Peak — churchkey · 2026-09-04
- Cerebras unveils CS-4 and WSE-3 Turbo, claiming 30x GPU inference speed — airesearch12 · 2026-09-04
- The Next Token Ep.05: Michelle Chen on AI inference at scale, open weights and burning money — ritakozlov · 2026-09-04