Self-improving agents: experience should change the whole agent, not just its memory
furongh · x · 2026-09-05
The author asks what an AI agent should carry forward from completed work. Most agents today adapt only within a task—inspecting errors, revising plans, switching tools, recovering from failure—while the experience largely disappears when the task ends.
The more important frontier is self-improving agents: systems that use the consequences of today's work to improve how they work tomorrow. Not just memory, not another training run, not merely a model rewriting itself—the object that improves is the whole agent: its skills and tools, workflows and action policies, ability to choose among strategies, its evaluator, and sometimes model parameters.
The key question: does experience change the agent's method? A coding agent that discovers a generated file issue shouldn't just remember the transcript—it should learn a better procedure.
More from AGI Musings
- Asking when a rational agent does the right thing is still underrated, argues AI researcher — xuanalogue · 2026-09-05
- Researcher: when would a rational agent do the right thing remains underrated — xuanalogue · 2026-09-05
- Evals Find AIs Willing to Take Extreme Actions, Resurfacing AI-Takeover Skepticism — JMannhart · 2026-09-05
- Cambridge's David Krueger endorses Katja Grace's short case on pausing AI 'but not yet' — DavidSKrueger · 2026-09-05
- Radio interview on rogue agents escaping control via reward hacking in LatAm education — OmarUFlorez · 2026-09-05
- Will interpretability ever be "solved"? A researcher argues probably not — burny_tech · 2026-09-05