Why specialized agents that update their own weights beat generalist models
willcb · x · 2026-09-26
The author argues that expecting models to be superhuman at everything lacks historical or theoretical precedent — the smartest humans are exceptionally specialized. A long-running adaptive agent harness is a primitive "specialized agent" limited by context length, filesystem expressivity, and bespoke retrieval; letting the agent change its own weights is strictly more powerful. The bitter-lesson version: use RL to train models to update themselves on-the-fly however they best see fit, just as compaction and subagent delegation are trained end-to-end today.
Related event: Researchers Debate Whether Task-Specific RL Can Produce General Agents(3 posts)→
More from AGI Musings
- "AI Safety Is Pseudoscience" Debate Hinges on OpenAI's Opaque Multi-Agent Training — basedjensen · 2026-09-26
- Railroads hit ~10% of GDP before demand existed — an AI bubble analogy — abhiadesai · 2026-09-26
- Measure model gaps in capability, not months — at the exponential, 3 months means something very different — maksym_andr · 2026-09-26
- UK Growth Survey Tool Compares AI Models' Answers Against Expert Economists — dc_lawrence · 2026-09-26
- Calling AI companies 'labs' is liability dressing, says founder selling agents — victor_explore · 2026-09-26
- Schmidhuber: AI has no moat, DeepSeek proved it, and the hype bubble should burst — SchmidhuberAI · 2026-09-26