Continual-ARC benchmark: top agents fail to learn from experience, a 9B fine-tune does
alifcoder · x · 2026-11-12
Unify AI launched Continual-ARC, a new benchmark testing whether agents actually get smarter from work they've already done, rather than just what they can do today. Results: OpenClaw, Hermes Agent and Prime Agent don't meaningfully learn from experience — none could memorize anything of value or generalize to new tasks. Training a small 9B model's weights, by contrast, achieved both memorization and generalization.
The takeaway: current agent frameworks' task completion ability and genuine learning-from-experience are separate things, and weight-level continual learning may be the real unlock.
Related event: Continual-ARC Benchmark: Mainstream Agents Fail to Learn from Experience(12 posts)→
More from coding & agent
- Choosing the Right Codex Model and Effort Level for Research Tasks, Plus an Open-Source Skill — udmrzn · 2026-11-12
- Who reads the queries your agent writes? A diagram of agentic search data flows — adnan_hashmi · 2026-11-12
- Hackathon project lets AI agents spend company money, but only within CFO-approved limits — kleffew94 · 2026-11-12
- In real orgs the model is rarely the problem — the context it gets is, says Salesforce engineer — msrivastav13 · 2026-11-12
- Salesforce Winter '27 adds AiAgentDefinition metadata for versioned agent deployments — msrivastav13 · 2026-11-12
- Solo dev builds zombie horde level with Tripo P2.0, Fable and new Kaigen engine — gdechichi · 2026-11-12