Sakana's PC-ALM Trains 1,000-Layer Networks Without Backprop, Within ~2 Points of SGD
demian_ai · x · 2026-09-15
Sakana AI's PC-ALM algorithm challenges backprop's monopoly: it trained 1,000-layer residual networks on MNIST using only local interactions between neighboring layers, staying within roughly two percentage points of backpropagation.
- The rewind-and-replay of errors through every layer is what keeps training in data centers; local learning signals could make on-device training practical.
- Mechanism: in vanilla predictive coding, supervision signals become too weak in deep, narrow networks to guide early layers — PC-ALM fixes this.
- The framing: a training algorithm is a bet on what computers should look like.
Related event: Sakana's PC-ALM trains 1000-layer networks without backpropagation(3 posts)→
More from Infra
- Helion closes oversubscribed $500M Series G for fusion energy — ycombinator · 2026-09-16
- nginx 1.31.6 Patches Heap Buffer Overflow in HTTP/3 (CVE-2026-90439) — jedisct1 · 2026-09-16
- AWS Trainium Runs PyTorch Natively, PyTorch Con Keynote Revealed — PyTorch · 2026-09-16
- A long-form explainer on why local inference matters — and why you need uncensored models — HankYeomans · 2026-09-15
- DGX Spark + TRELLIS.2 turns pencil sketches into 3D GLBs fully locally — jasonkneen · 2026-09-15
- Skipping a second RTX 5090 for two more Sparks: local inference user explains why — ideamaker321 · 2026-09-15