Meta Shares PyTorch Tools to Keep Long Training Runs from Failing: torch_remat, FlexShard
lmoroney · x · 2026-10-21
- At PyTorch Conference North America, Meta's Alban Desmaison and Edward Z. Yang gave a keynote on keeping large training runs from failing after you've already paid for the GPUs.
- Highlights:
- torch.accelerator and backend testing for simpler, more reliable hardware integration.
- torchremat for activation checkpointing.
- FlexShard, which decouples storage sharding from compute sharding.
- Practical advice: before your next long run, do a short smoke test on the exact hardware and config you plan to use, and fix whatever breaks there first.
Related event: Meta Engineers Unveil Training Stability Tools at PyTorch Conference(3 posts)→
More from Infra
- vLLM's specialized decode path hits 829 P90 TPS per user on four B300 GPUs, 26% faster — vllm_project · 2026-10-21
- vLLM says it can reproduce DeepSeek's inference performance, yielding $39/GPU/hr revenue — vllm_project · 2026-10-21
- vLLM matches DeepSeek's DeepInfer decode performance within 6%, generating $39/GPU/hr at off-peak pricing — vllm_project · 2026-10-21
- AMD brings EAGLE-4 speculative decoding to Instinct MI355X day 0, 1.6x faster than built-in MTP — hongyangzh · 2026-10-21
- TokenGate: learned token pruning speeds up LLM prefill up to 11.3x — Liana Patel · 2026-10-21
- Modal revives multiplayer game Curvytron to demo its new Sticky Sessions routing — charles_irl · 2026-10-21