Nightly training runs as pseudo continual learning: still cumbersome, costly, and power hungry

PTrubey · x · 2026-09-06

The author revisits his prediction from a year ago: since models lack continual learning, the industry could paper over it with compute — running nightly training runs akin to human sleep consolidation. Post-Claude progress in scaffolding and agents has proven him right.

But this approach remains cumbersome, expensive, and power inefficient; AI robotics will expose the weaknesses further
Companies are still researching no-backprop continual learning algorithms that could be orders of magnitude faster, cheaper, and more power efficient
The holy grail: teaching a robot a new skill by talking to it, maybe with demos. He once thought this needed a new architecture (e.g., Rain Neuromorphic's approach), but now suspects backprop plus fancy scaffolding and a tight train/inference loop might suffice
His conclusion: we're only in the first inning of AI advancement — more shoes are waiting to drop

Original post →

Nightly training runs as pseudo continual learning: still cumbersome, costly, and power hungry

More from AGI Musings