PyTorch 2.13 Released: FlexAttention on Apple Silicon, 4× Peak Memory Cut for Large Vocab Models
PyTorch · x · 2026-08-01
PyTorch 2.13 has been officially released, bringing several key performance optimizations and feature expansions:
- Hardware & Memory: FlexAttention now supports Apple Silicon. The introduction of nn.LinearCrossEntropyLoss cuts peak memory by up to 4× for large-vocabulary models.
- Compute Acceleration: FlexAttention compiles kernels specialized for requested masking patterns to skip unnecessary computation, achieving up to 12× speedups over SDPA in tested sparse configurations. It also introduces a deterministic backward path on CUDA, removing a source of gradient non-determinism.
- Ecosystem: Updates across distributed training, compilation, profiling, and on-device inference.
A live Q&A session also covered CUDA version support, CuTeDSL, Python 3.15, and plans for PyTorch 2.14.
Related event: PyTorch 2.13 Released: FlexAttention Hits Apple Silicon(2 posts)→
More from Infra
- a16z: AI Infrastructure Demand Shows No Signs of Slowing Amid Supply Chain Snags — a16z · 2026-08-01
- Together AI Deep Dive: Autoscaling Endpoints for LLM Inference — togethercompute · 2026-08-01
- Vercel AI Gateway Adds Team and Project Spend Budgets — cramforce · 2026-08-01
- Tesla Signs 469MW Solar Deals to Lock in AI Compute Power Years Ahead — XFreeze · 2026-08-01
- Local Deployment on DGX Spark: Exploring Upgrades Beyond Qwen 3.5 122B — Voxandr · 2026-08-01
- Analyst Spots Equinix Expanding San Jose Campus by ~200MW — BenBajarin · 2026-08-01