Sakana AI's PC-ALM trains 1000-layer nets with local learning, no backpropagation
SakanaAILabs · x · 2026-09-16
Sakana AI introduced PC-ALM (Augmented Lagrangian Predictive Coding), a local-learning alternative to backpropagation that trains 1000-layer neural nets using only layer-local dynamics.
- Method: builds on predictive coding by accumulating per-layer constraint errors into layer-local Lagrange multipliers, aligning weight updates toward BP while keeping PC's inference budget.
- Results: matches BP performance across all width-depth regimes up to depth 128, notably fixing PC's weakness in deep narrow networks.
- Mechanism: shows "ballistic" credit propagation with signals evenly distributed across layers, vs. PC's slow diffusive spread.
- Implication: the augmented Lagrangian framework generalizes PC and may explain how distributed systems could propagate BP-like credit signals via purely local dynamics.
Related event: Sakana AI Unveils PC-ALM: Training 1000-Layer Nets Without Backprop(5 posts)→
More from Research
- NVIDIA's Kumo Is a Family of Four Tabular Foundation Models, Not Just One — pandeyparul · 2026-11-02
- Aaroth shares Agreement Theorem lecture: round-wise calibration achievable without distributional assumptions — Aaroth · 2026-11-02
- Economist Aaroth: fast agreement dynamics survive without Bayesian updates or accurate priors — Aaroth · 2026-11-02
- ELLIS names 160 new ML Fellows and Scholars for 2025–2026 — JJitsev · 2026-11-02
- Meme turn 50: Aumann's 'Agreeing to Disagree' paper anniversary — Aaroth · 2026-11-02
- Anthropic: analysts unknowingly adopted their agent's secret lean 86% of the time — shoumikgoswami · 2026-11-02