AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?
Dong Yan, Jian Liang, Dapeng Hu, Ran He, Nicholas Jing Yuan, Qi Zhang, Tieniu Tan
cs.AI, cs.LG
2026-08-01
AgentStream streams six agent benchmarks to test five self-evolving methods on three frontier models; in about a third of configurations self-evolution underperforms the vanilla baseline, and the weakest model loses ground in every scenario.
Self-evolution has become a popular idea for LLM agents: after finishing a task, the agent distills the interaction into experience (a sharper prompt, a memory entry, a reusable skill), stores it, and draws on it for later tasks. The premise is that accumulated experience compounds, so the agent gets better the more it runs.
Nearly all existing evaluations are "independent": each task is solved in isolation, no state carries over, and scores are averaged. The few studies that move toward streaming do so on a single benchmark under a single evolution component. Real deployment does not look like that. Tasks arrive back to back and across domains; you finish a coding job, then a deep-research query, then a multi-turn conversation. Whether experience helps or just adds noise in that regime has gone unmeasured.
AgentStream measures it.
AgentStream organizes six mainstream agent benchmarks (AppWorld interactive coding, BFCL multi-step function calling, BrowseComp+ deep research, HLE expert-level reasoning, SWE-bench Verified software engineering, Tau2 dual-control dialogue) into a configurable task stream, and instantiates three streaming scenarios at test time, each widening the scope and domain mix:
Across these scenarios the authors combinatorially evaluate five self-evolving methods covering different evolution components: ACE (evolves the context prompt), A-Mem (Zettelkasten-style memory linking), ReasoningBank (distills reasoning strategies from successful and failed trajectories), AutoSkill (extracts reusable skills), and Harness (jointly updates prompt, skills, and memory). Three frontier models serve as base: GPT-5.4-medium, Gemini 3.1 Pro-medium, Claude Opus 4.7-high. Every configuration is compared against a vanilla baseline that does no self-evolution, isolating the net gain from experience.
The headline is a cold shower: self-evolution is not uniformly beneficial and underperforms the vanilla baseline in a substantial share of configurations.
Per-scenario aggregate (45 = 3 models x 5 methods configurations):
| Scenario | Positive (> Vanilla) | Avg. gain | Top-1 rate |
| Isolated | 34/45 (75.7%) | +1.37% | 38% |
| Sequential | 28/45 (62.3%) | +0.75% | 29% |
| Interleaved | 28/45 (62.3%) | +0.90% | 33% |
Isolated is the most reliable: experience stays within a single domain, so it least often gets in the way. Once the stream gets mixed, the positive rate drops to just above 60%.
The benefit is gated by base-model capability. The weakest model, GPT-5.4, shows negative average gains across all three scenarios (from -0.35% to -0.78%), and only A-Mem turns positive there at all. The authors attribute this to a bootstrap loop: experience quality depends on task success, so a weak model that fails tasks stores noise that drags down later ones. Gemini 3.1 Pro and Claude Opus 4.7 are the two that consistently benefit.
A counterintuitive wrinkle is that the gain is non-monotonic in model strength. The mid-capability Gemini sometimes gains more than the stronger Claude, so a stronger base does not automatically mean self-evolution pays more.
Cost splits by model too. On Gemini 3.1 Pro, ReasoningBank delivers a +2.3% gain while cutting cost to 64% of the baseline, the best deal. On GPT-5.4, A-Mem's +3.2% gain costs 577% of baseline. Method choice is tightly bound to the model: A-Mem leads on GPT-5.4, ACE leads on Gemini, and rankings shift on the third. No single method dominates across all models and scenarios.
For anyone building agents, the value here is not an endorsement of any one method. It removes a default assumption. Treating self-evolution as a switch you flip on to get a smarter-over-time agent is not supported by the numbers. Whether to deploy it, and which method to use, depends on three things: whether the base model is strong enough (weak models get worse), whether tasks stream across domains (the more mixed, the more you risk interference), and the method-model match (the same method can rank first on one model and third on another).
There are usable takeaways. For cost-sensitive setups on a weaker model, retrieval-style methods like ReasoningBank that also cut cost are the safer bet. For cross-domain interleaved streams, A-Mem, which retrieves and suppresses interference, stays positive across all three scenarios with low variance.
The authors flag two. First, "model capability strength" is an empirical observation under their specific setup (the Exgentic framework, their prompting, their benchmark mix), not a universal ranking; different frameworks, prompts, or benchmarks could reorder it. Second, only three scenarios and six benchmarks are covered, representative but not exhaustive.
Two more caveats deserve weight. The absolute gains are small, mostly 1 to 2 percentage points, and the authors note the results rest on a single evaluation, so whether that movement clears the noise floor is not nailed down. And the two headline claims, that benefit is gated by capability and non-monotonic in strength, rest on only three data points, a thin base for a general rule. Treat them as an empirical hint rather than a settled conclusion.