DeepMind's Mathieu Blondel maps the 'local losses' framework unifying modern AI training

mblondel_ml · x · 2026-10-24

Google DeepMind researcher Mathieu Blondel argues a key ingredient behind modern AI is decomposing long sequential computation into many independent, local supervised learning problems — with variants differing mainly in where local targets come from. His taxonomy:

• Targets from data/expert (imitation): teacher forcing, behavior cloning, step-level trace supervision; failure mode is exposure bias, fixed via scheduled sampling or DAgger
• Targets from designed corruption/transport: diffusion (denoising score matching), flow matching/rectified flow, denoising autoencoders and masked modeling (BERT, MAE)
• Targets bootstrapped from the model (self-consistency): TD learning, GFlowNets, consistency models
• Targets from search or hindsight: Expert Iteration/AlphaZero, hindsight replay, rejection-sampling fine-tuning/STaR, process reward models
• Targets as free variables: multiple shooting/collocation, ADMM training, target propagation
• Local by architecture or physics: physics-informed losses, layer-local training (forward-forward, synthetic gradients), representation prediction/distillation (BYOL, JEPA)

The chart unifies seemingly disparate lines — RL, diffusion, process rewards, JEPA — under one framework.

Original post →

More from Research

• Action learning: score whole trajectories with no forces or simulator, 2000× lower error — LeoTZ03 · 2026-10-24
• BeNice! method shares KV-cache across loops, scaling looped LLM depth without extra memory — stochasticchasm · 2026-10-24
• 215-state Turing machine proven to halt iff ZFC is inconsistent, verified in Metamath Zero — teortaxesTex · 2026-10-24
• OmniRay from JHU hits SOTA camera-agnostic monocular 3D reconstruction from one image — kwangmoo_yi · 2026-10-24
• AWS multi-turn RL guide: $339 lifts gpt-oss-20b Wordle solve rate 61%→84% — AWS ML Blog · 2026-10-24
• Eric Horvitz spotlights Stanford's tool-making research philosophy behind optogenetics — erichorvitz · 2026-10-24

Research channel →