Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100
sguada · x · 2026-10-27
While a well-funded startup raised $1B to build megakernels for data-efficient learning, independent researcher deepaksinghml open-sourced Latch: a compiler that generates training megakernels directly from PyTorch.
Published H100 benchmarks (vs compiled PyTorch):
- Time-to-loss: TinyMoE (batch 32×1024) reaches loss 3.30 in 857s vs 1432s — 40% less time.
- Training updates: TinyMoE (batch 2×1024) at 13.6–13.8ms vs 37.5–38.4ms per step, 2.75x faster.
- Test-time adaptation: GradLev (1024 steps) at 54ms vs 190ms per step, 3.5x faster (baseline uses CUDA graphs).
Latch is open source and aims to automate megakernel generation so small teams can do kernel-level training optimization without a $1B war chest.
More from Infra
- AI Infra Meme: While Countries Need TW-Scale Power for Datacenters, People Argue About Lightbulbs — mgostIH · 2026-10-27
- Legora runs 2,000 parallel agent evals on Modal Sandboxes for legal AI — blelbach · 2026-10-27
- Cloud release velocity scores: AWS 95, Azure 90, Google Cloud 82 — DavidLinthicum · 2026-10-27
- Selling a 5090 to scale local inference: one business weighs 5 hardware paths — Anarchaotic · 2026-10-27
- AWS tutorial: fleet-wide edge CV model lifecycle with SageMaker SDK v3, ~$0.07 per run — AWS ML Blog · 2026-10-27
- Qwen3.8-27B hits 40 tok/s on Jetson Thor with INT4 + MTP, accuracy beats FP16 — No-Dragonfly6246 · 2026-10-27