kalomaze: Modern DNI May Approximate the Regimes Where Backprop Fails
kalomaze · x · 2026-09-30
kalomaze argues that ES/genetic-algorithm training is "Sakana-tier nerdsnipe" — if you abandon backprop, do it for regimes where naive backprop is intractable without BPTT. His take: there's an unobservable regime we can't train directly; DNI loses to full backprop where the latter is tractable but behaves close to it; and a modern, especially distributional, form of DNI could yield a fairly faithful approximation of that unobservable regime.
More from Research
- SEMA trains open-source models for multi-turn adversarial attacks without human scripts — burkov · 2026-09-30
- Artificial Analysis breaks down individual evals in Intelligence Index v4.3.2 — ArtificialAnlys · 2026-09-30
- QuAcc open-source library unifies atomistic simulation workflows for the AI era — rbhar90 · 2026-09-30
- Adobe's TAC timestamped audio captioning model accepted at NeurIPS, hits SOTA — justin_salamon · 2026-09-30
- NVIDIA/MIT's Physis-Lang tops Physics-IQ leaderboard, +5.5 points via evolved captions — songhan_mit · 2026-09-30
- LeCun co-authors AD-E2E-JEPA: world model drives end-to-end without any driving policy — ylecun · 2026-09-30