PyTorchCon talk: QR, eigendecomposition and Cholesky kernels decide if better optimizers actually train faster
PyTorch · x · 2026-10-22
A PyTorchCon 2026 talk (via the official PyTorch account) digs into the linear algebra kernels behind better optimizers — fewer training steps don't necessarily mean faster training. Focusing on QR, symmetric eigendecomposition, and Cholesky, and drawing on GPU MODE's linear algebra competition, it covers how sequential dependencies constrain GPU execution, how implementations expose parallel work and overlap independent stages, and how combining updates reduces data movement. Full video available.
More from Infra
- Meta plans up to $145B in capex for 2026, more than double 2025 — Polymarket · 2026-10-22
- pi ships rec inference engine, runs locally on M5 Max in two lines — _rockt · 2026-10-22
- Researcher warns GitLab flaw is 'really bad,' urges users to patch now — basedjensen · 2026-10-22
- PyTorch Con Demo Theater: 13 talks including Wan 2.2 video generation sped up 53.6x — PyTorch · 2026-10-22
- TorchTitan adds 5D parallelism for DeepSeek-V4, Qwen3.5, Kimi K3; Muon optimizer 79% faster — PyTorch · 2026-10-22
- JNTC cuts TGV glass substrate copper filling from a day to minutes, targets 2027 mass production — toptickcrypto · 2026-10-22