Sakana's PC-ALM Trains 1,000-Layer Networks Without Backprop, Within ~2 Points of SGD
demian_ai · x · 2026-09-15
Sakana AI's PC-ALM algorithm challenges backprop's monopoly: it trained 1,000-layer residual networks on MNIST using only local interactions between neighboring layers, staying within roughly two percentage points of backpropagation.
- The rewind-and-replay of errors through every layer is what keeps training in data centers; local learning signals could make on-device training practical.
- Mechanism: in vanilla predictive coding, supervision signals become too weak in deep, narrow networks to guide early layers — PC-ALM fixes this.
- The framing: a training algorithm is a bet on what computers should look like.
Related event: Sakana AI Unveils PC-ALM: Training 1000-Layer Nets Without Backprop(5 posts)→
More from Infra
- Big Tech's Future Data Center Leases Top $1.3T, 70% Added in Last Four Quarters — rohanpaul_ai · 2026-11-02
- Daniel Lemire: AI is exploding demand for boring compute, server CPUs to double — lemire · 2026-11-02
- Physical AI's real story: a 4-layer map of 15 companies behind humanoid robots — pdamodaran · 2026-11-02
- Under 5% of US consumers pay for AI, yet compute demand is already forcing usage cuts — NinaDSchick · 2026-11-02
- Sam Altman: AI query energy cost fell 1,000x-100,000x in 3-4 years — rohanpaul_ai · 2026-11-02
- Marvell and Lumentum to merge, targeting $550B AI connectivity market — BenBajarin · 2026-11-02