DeepSpeed details AutoSP, Muon optimizer, and AutoTP/AutoEP advances at PyTorchCon
PyTorch · x · 2026-10-22
At PyTorch Conference 2026, Zhipeng Wang (Google) presented DeepSpeed's latest training-systems co-design work: AutoSP for automatic sequence parallelism, Muon optimizer support, and AutoTP/AutoEP for Hugging Face models, plus community efforts with hardware vendors and AI infra partners to support emerging large-scale workloads.
More from Infra
- Ollama Runs Open Models on NVIDIA Blackwell via Crusoe, Live in Two Weeks — ollama · 2026-10-22
- Claim: every 2nd/3rd-tier inference provider now has a recipe for 90%+ compute margins — zephyr_z9 · 2026-10-22
- NVIDIA keynote: silicon photonics CPO switches hit 102.4 Tb/s, Vera Rubin architecture unveiled — thealexbanks · 2026-10-22
- PyTorch Foundation Adds Nine Members, Nebius, Nscale and Shopify Join as Platinum — PyTorch · 2026-10-22
- GPU startup Flappy Airplanes raises $1B Series A co-led by Index and Lux — vitaliychiley · 2026-10-22
- Crusoe unveils Tinker API-compatible LLM post-training platform at PyTorch Conference — PyTorch · 2026-10-22