New Continual-ARC benchmark: OpenClaw, Hermes, Prime agents fail to learn from experience
hey_abusiddik · x · 2026-11-11
Unify's new Continual-ARC benchmark finds that OpenClaw, Hermes Agent and Prime Agent don't meaningfully learn from experience — none memorize anything of value or generalize to unseen tasks.
Daniel Lenton's advice to founders: when an agent claims it gets better with use, ask how it performs on work it has never seen — recalling old work is easy to fake. Notably, training a small 9B model's weights achieved both memory and generalization.
More from coding & agent
- Cloudflare's weekend-trained hybrid model cuts agent cost from 40K to 1.8K tokens — TheTuringPost · 2026-11-12
- Ex-Meta CV researcher builds AdaL v2, a chat-driven video agent harness for GTM — Scobleizer · 2026-11-12
- chrisalbon: Show me 3 months and 1000 prompts, not one magic prompt — chrisalbon · 2026-11-12
- Agent autonomously replaces Gazebo's Ogre rendering engine with Filament in 36 hours — facontidavide · 2026-11-12
- Guardian: open-source authority layer pins human approval to each exact agent tool call — manofstyle04 · 2026-11-12
- Sentry open-sources tattletale: trace Claude Code and Codex agent sessions for token inefficiency — zeeg · 2026-11-12