Continual-ARC benchmark: top agents fail to learn from experience, a fine-tuned 9B model succeeds
alifcoder · x · 2026-11-12
Unify.ai's new benchmark Continual-ARC tests whether agents can learn from experience. The result is a wake-up call: OpenClaw, Hermes Agent and Prime Agent showed no meaningful memorization or generalization to new tasks, while training a small 9B model's weights achieved both. Daniel Lenton argues the next breakthrough may be agents that actually learn, not just remember.
More from coding & agent
- Together Compiler: agent-driven continual compilation for AI inference — togethercompute · 2026-11-12
- Kalshi partners with AnyAlpha to let AI agents trade prediction markets 24/7 — ycombinator · 2026-11-12
- Google solicits Kubernetes users to help shape Agent Substrate for Managed Agents — rakyll · 2026-11-12
- Building apps in the agent era: elegant abstractions still beat raw code generation — cnakazawa · 2026-11-12
- Grok agent autonomously browses, orders and pays for a book on an unfamiliar site — xkonjin · 2026-11-12
- Stop Managing Sandbox Fleets for Cloud Agents, Argues gakonst: Go Harness-First — andreisavu · 2026-11-12