PyTorch 2.14 Ships: New nccl2 Backend, First-Class Fault Tolerance, NVFP4 GEMM
StasBekman · x · 2026-09-04
PyTorch 2.14 is out with 2,995 commits from 487 contributors. Highlights: a new nccl2 distributed backend ported from torchcomms with nonblocking communicators; fault tolerance as a first-class c10d concept with in-place process-group reconfiguration and a backend-agnostic Flight Recorder; NVGEMM bringing CuTeDSL-generated CUTLASS kernels with NVFP4 GEMM to Inductor; native linear algebra on Apple Silicon; declarative dynamic shapes via @dynamicspec; experimental torch.compile support for complex tensors; plus support for Rubin (sm107), ROCm 7.14, and Intel XPU graph capture.
Related event: PyTorch 2.14 Released with 2,995 Commits from 487 Contributors(3 posts)→
More from Infra
- Auction-based LLM routing paper finds fixed token pricing wastes 10%-71% in costs — _onionesque · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- SGLang Summit sponsors jump from 11 to 36 in one week — ying11231 · 2026-10-18
- Oracle CDS hits record 300bps, called first AI trade domino after Barclays sell downgrade — arthurcolle · 2026-10-18
- Agent Coding Workflow Would Cost ~$40k/Month on API Pricing, Dev Reveals — ChrisRackauckas · 2026-10-18
- PS5LM: llama.cpp port turns a jailbroken PS5 into a home LLM server (500+ GitHub stars) — wateriscoding · 2026-10-18