Ex-OpenAI's Yaroslav Bulatov on why backprop should go: it's the memory wall
yaroslavvb · x · 2026-10-24
Yaroslav Bulatov, who worked at Google Brain, early OpenAI, Meta, and Together AI, explains on The Interlude Show why he wants to get rid of backpropagation: the memory wall. Data movement dominates energy use, and the memory wall is reshaping algorithm design.
Key points:
- Gradient checkpointing can make a workload faster even while doing more computation;
- He now runs the Sutro Group at South Park Commons, exploring energy-efficient learning algorithms with AI agents;
- The conversation covers early OpenAI, his skepticism about reinforcement learning, and why recursive self-improvement still faces new bottlenecks.
More from Research
- Anthropic uses Claude to formally verify 100+ open-source libraries in weeks — pzakin · 2026-10-24
- Agent S4 evolves interfaces to let open 27B model beat GPT-5.5 on OSWorld V2 at $1.10/task — xwang_lk · 2026-10-24
- UBC NLP lands 21 papers at EMNLP 2026 spanning culture, RAG, and agents — mageed · 2026-10-24
- Terminal-Bench exposes NVIDIA Nemotron 3.5 near-zero score vs Kimi K3.1's 65.7% — JJitsev · 2026-10-24
- Kimi K3.1 Pro reportedly first Chinese model to solve 20x20 nonograms — fairydreaming · 2026-10-24
- Self-Evolving Benchmarks: Multi-Agent Evolutionary Search Auto-Generates Challenging Tool-Use Tasks — MichaelD1729 · 2026-10-24