Will DePue Argues Continual Learning Is Not a Real AGI Bottleneck
willdepue · x · 2026-08-13
AI founder Will DePue shared a lengthy thread arguing that 'continual learning' is not a single defined roadblock to AGI, claiming most limitations are overstated.
- Catastrophic Forgetting: He views this as purely an engineering 'skill issue' that is non-existent with competent methods.
- Technical Alternatives: If infinite memory is the goal, RL and scratchpads++ will solve it. For better in-context learning, simply scaling up models already helps.
- Core Argument: The only solid point is the probable need for gradients and weight adaptation at test time, given their power during training. However, this still doesn't justify it as a true AGI bottleneck. In a follow-up, he dismissed the concept as devolved ML slang.
More from AGI Musings
- AI Didn't Make Software Engineering Easier, Just Differently Difficult — HanchungLee · 2026-08-13
- High Compute Costs Mean Top ML Talent Is Likely Stuck in Rural Areas — DrDatta_AIIMS · 2026-08-13
- Top AI Professor Toby Walsh to Deliver Lecture on the Real Impact of AI on Jobs — TobyWalsh · 2026-08-13
- Goldman Warns of 'Google Zero': AI Search Threatens Publishers with 40% Traffic Drops — SumitGup · 2026-08-13
- Watermarking AI Outputs: A Key Defense Against Recursive Training — lilyraynyc · 2026-08-13
- Call for Humanities: We Need the Frankfurt School to Unpack AI — joshua_saxe · 2026-08-13