PyTorch 2.13 Brings FlexAttention to Apple Silicon, Cuts Peak Memory by 4×
PyTorch · x · 2026-08-01
PyTorch 2.13 has been released with several major highlights:
- FlexAttention on Apple Silicon: Previously CUDA-focused, this feature now supports Apple Silicon.
- Memory Optimization: The new nn.LinearCrossEntropyLoss cuts peak memory by up to 4× for large-vocabulary models.
- Significant Speedups: FlexAttention compiles kernels specialized for the requested masking pattern to skip unnecessary computation, achieving up to 12× speedups over SDPA in tested sparse configurations, with a deterministic backward path on CUDA.
- Other Updates: Improvements across distributed training, compilation, profiling, and on-device inference.
The live Q&A also covered CUDA version support, CuTeDSL, Python 3.15, and plans for PyTorch 2.14.
Related event: PyTorch 2.13 Released: FlexAttention Hits Apple Silicon(2 posts)→
More from Infra
- Dassault Systèmes and NVIDIA Accelerate Engineering Simulation with AI & GPUs — NVIDIA Developer · 2026-08-01
- Cerebras' Ultra-Fast Inference Wins Praise: 'No Going Back' Once Tried — soumitrashukla9 · 2026-08-01
- Local Open-Source LLMs Now Match Frontier Performance from 2 Months Ago — ccerrato147 · 2026-08-01
- The 'Diffusion Lag': Sub-4B Edge Models to Match 2026 Frontier LLMs by 2028 — almostsweet · 2026-08-01
- Meta Engineer Shares MLSys Keynote: Using AI to Liberate Systems Researchers — salykova_ · 2026-08-01
- Building a Local RAG Stack on Mac Studio: Read/Write Separation and MCP — Joules_Jokes_Leks · 2026-08-01