Continual-ARC benchmark: leading AI agents fail to learn from experience
nikola_mr64990 · x · 2026-11-12
Unify.ai released Continual-ARC, a new benchmark testing whether agents genuinely learn from experience. Results: OpenClaw, Hermes Agent and Prime Agent "don't meaningfully learn from experience" — none could memorize anything of value or generalize to new tasks, while training a small 9B model's weights achieved both.
Commentators note the distinction isn't just remembering information, but knowing what's worth keeping and applying to new problems. A quantifiable test for true continual learning, relevant to researchers and agent builders.
More from coding & agent
- Stanford NLP course's best homework: build your own agent harness — stanfordnlp · 2026-11-12
- Anthropic details agent memory: dreaming organizes Claude's memory, wiki+search hits 61% — RLanceMartin · 2026-11-12
- Talk preview: scaling 1,000 Fable 5.2 agents to formalize Fermat's Last Theorem — RLanceMartin · 2026-11-12
- Opinion: AI infra is becoming a coding runtime as intelligence units outgrow tokens — prateekj · 2026-11-12
- Claude Docs/Slides block external MCP clients: no public API, export only — steipete · 2026-11-12
- Three AI game dev competitions land this month, one with a $47,800 prize pool — Promptmethus · 2026-11-12