New Continual-ARC Benchmark Finds Leading AI Agents Fail to Learn From Experience
alifcoder · x · 2026-11-12
Unify.ai 团队发布新基准 Continual-ARC,测试 AI agent 能否真正从过往经验中学习,结果令人失望:
- OpenClaw、Hermes Agent、Prime Agent 三个主流 agent 均无法有效从经验中学习——既记不住有价值的信息,也无法泛化到新任务
- 作者指出「记住上次发生了什么」不等于「从中学到东西」:agent 能保存对话和指令,但面对未见过的问题时并不会真的变强
- 作为对照,对一个小型 9B 模型做权重训练则同时实现了记忆与泛化
作者呼吁社区出现更多专门测试 agent 持续学习与泛化能力的基准。
More from coding & agent
- Plurai launches free simulation plugin that digital-twins your AI agent before production — nikola_mr64990 · 2026-11-12
- An agent saying "Done" isn't done: Plurai verifies actual agent behavior — nikola_mr64990 · 2026-11-12
- LlamaIndex ran a 4-day agentic pen test on LlamaParse and shared 4 key lessons — llama_index · 2026-11-12
- Manhattan v0.0.1: an open-source font where every letter is a Python function — allanyu_ · 2026-11-12
- Backplane lets you prompt custom expansion cards into existence — _Stocko_ · 2026-11-12
- Coding agent runs 51 minutes burning 248k tokens in a single session — _Stocko_ · 2026-11-12