PyTorch's new flex_gemm API fuses GEMM with epilogues written as plain PyTorch functions
tri_dao · x · 2026-10-23
Tri Dao boosted drisspg's PTC talk on flexgemm, a new gemmepilogue API in PyTorch built on Quack. It lets you fuse a GEMM with an epilogue written as an ordinary PyTorch function: a single kernel computes epilogue(torch.mm(a, b)), running the epilogue on the accumulator in registers before anything is stored.
- Open-sourced in the transformernuggets repo with 9 example scripts: pointwise, multiple outputs, SwiGLU, reductions, MXFP quantization, backward, stochastic rounding, function transforms, and autotuning.
- Each script validates correctness against plain torch.compile and prints GPU times for both paths.
- Author plans to bootstrap the effort and keep adding examples.
More from coding & agent
- 35 CVEs tied to AI coding tools this year, with no review budget to match — YvesMulkers · 2026-10-23
- Indie dev's agent tracks 519 directories, 311 submitted for SEO backlinks — fekdaoui · 2026-10-23
- ScienceClaw Hackathon at MIT Media Lab to build decentralized AI agent collectives for science — ProfBuehlerMIT · 2026-10-23
- What should a native agent harness handle beyond tool calling? — Glittering812 · 2026-10-23
- Running ~100 DeepSeek v4.1 flash agents in parallel for under $20/month — zack_overflow · 2026-10-23
- MCP's stateless 2026-07-28 spec drops session-id — how are stateful tools adapting? — Stock-Pumpkin-8859 · 2026-10-23