Torch Compile underrated: CUDA Graphs + PDL hits 98% of megakernel speed on Llama 1B
simran_s_arora · x · 2026-11-17
Another thread from Dylan Lim: Torch Compile's generality and performance are underappreciated — default Inductor kernels, CUDA Graphs, and fine-grained PDL placement reach 98% of their previous Llama 1B megakernel performance.
Chart data on Llama 1B / H100 / batch size 1: PyTorch eager 26 TPS → torch.compile 669 → +CUDA graphs 864 → +PDL 953 → +fine-grained PDL 975 TPS (megakernel: 990 TPS). But Torch Compile has fundamental shortcomings: compiler heuristics can't reliably pick the best implementation across workloads, isolated kernel measurements don't reflect e2e speed, and strict rules leave performance on the table.
More from coding & agent
- A one-line prompt fix for "AI brain fry": limit focus to 1-3 items — gerardsans · 2026-11-17
- Park, a Canvas-Based Mac Browser From Arc's Lineage, Launches Public Beta — round · 2026-11-17
- Vision AI watches a garage door for ~$1.50/year, replacing a hardware sensor — SuB8u · 2026-11-17
- Ex-fintech insider: agentic payments UX is an open market waiting to be owned — SimonBalmain · 2026-11-17
- He automated his writing with a Claude Skill — then used it to write about itself — danshipper · 2026-11-17
- PostTrainBench one year on: AI agents now run complex post-training pipelines — maksym_andr · 2026-11-17