NVIDIA touts PyTorch's 50M+ monthly CUDA downloads with a decade of optimizations
NVIDIAAI · x · 2026-10-20
Ahead of PyTorch Conference 2026, NVIDIA celebrated its partnership with PyTorch, which now sees 50+ million monthly downloads with CUDA support. Shared timelines cover milestones from the CUDA backend in 2016 through FSDP, FP8 training, Triton GPU kernels in PyTorch 2.0, and FlexAttention/FlashAttention-4 on Blackwell, plus blogs showing MXFP8 MoE training 1.3x faster than BF16 and up to 41% faster pre-training for DeepSeek-V3 on B200 with TorchTitan.
More from Infra
- Cloudflare cut its AI code reviewer's inference bill by 68% with prompt caching and model routing — DanielLockyer · 2026-10-20
- DeepSeek launches DeepInfer with open prefill/decode benchmarks and reproducible scripts — Xianbao_QIAN · 2026-10-20
- Auto-research on idle GPUs: a CoreWeave pipeline that keeps agents busy — _ScottCondron · 2026-10-20
- Huawei's 960DT chips: most 2027 supply reportedly earmarked for DeepSeek supercluster — teortaxesTex · 2026-10-20
- €550 Used HP Z640 With Dual RTX 3060s: Reader Asks for MoE Local LLM Build Sanity Check — ElekDn · 2026-10-20
- NVIDIA GTC Berlin kicks off with Jensen Huang keynote on October 21 — IanAndrewsDC · 2026-10-20