Ex-OpenAI exec: Transformer architecture is the new learning bottleneck
_arohan_ · x · 2026-08-20
Jerry Tworek, Co-Founder of Core Automation and former VP at OpenAI, discusses why scaling RL hasn't delivered AGI. Despite benchmark improvements, real work remains undone. He diagnoses the Transformer architecture itself as the bottleneck, noting that models stop learning once deployed. The interview covers model capability limits and future directions.
More from AGI Musings
- "An LLM doesn't understand anything" gets the perfect parody reversal: neither do you — VoidStateKate · 2026-10-16
- Nina Schick: Europe's Policy Circle Still Doesn't Believe FSD, Orbital Compute Are Real — NinaDSchick · 2026-10-16
- Meta's Superintelligence Lab Runs 200+ Direct Reports—A Lesson for Flat Multi-Agent Systems — peterjliu · 2026-10-16
- Andy Masley lays out his version of effective altruism, pluralism over utilitarian math — RobbWiller · 2026-10-16
- We're forgetting how much of a miracle GPT-3.5-level intelligence was, and it may cost us — EigenGender · 2026-10-16
- Ex-Anthropic's McDonagh Builds Compute Intelligence Agency to Study Frontier AI Costs — McDonaghMatthew · 2026-10-16