Sakana's PC-ALM trains 1000-layer nets without backprop; LeCun calls it target prop reborn
ylecun · x · 2026-09-15
Sakana AI introduces PC-ALM, a local-learning alternative to backpropagation that trains 1000-layer neural nets using only local dynamics.
- Each layer's input acts as a free latent variable serving as a target for the previous layer — a "target prop" approach
- It can be derived from an augmented Lagrangian formulation of backprop, turning layer-to-layer consistency constraints into penalties
- Yann LeCun welcomed the revival of an idea his lab explored with sparse auto-encoders in the late 2000s, but noted target prop optimizes the same criterion as backprop — only the gradient evaluation differs, perhaps more biologically plausible
Related event: Sakana AI Unveils PC-ALM: Training 1000-Layer Nets Without Backprop(5 posts)→
More from Research
- General Reasoning explains how to train agent swarms with RL and latency pressure — rosstaylor90 · 2026-11-02
- Mathematician Tony Feng: current-paradigm LLMs lack the diversity of thought research needs — littmath · 2026-11-02
- Measure Theory vs Probability: 'You're Confusing the Finger With the Moon' — LucaAmb · 2026-11-02
- DStill Distills Multi-Step Denoising Into One Pass, 6.45x Speedup for Diffusion Speculative Decoding — LucaAmb · 2026-11-02
- Radial gate objective cuts diffusion transformer FID from 16.24 to 11.83 — Junha Park · 2026-11-02
- REA: async brain-cerebellum runtime for robots where policies never wait on the LLM — Pengfei Zhou · 2026-11-02