New Continual-ARC benchmark: leading agents fail to learn from experience, a 9B fine-tune succeeds
Med1_Ai · x · 2026-11-11
Unify.ai's new Continual-ARC benchmark tests whether AI agents can learn from sustained real-task experience. The results are sobering:
- OpenClaw, Hermes Agent, and Prime Agent memorized nothing of value and failed to generalize to new tasks
- As a contrast, training a small 9B model's weights achieved both memorization and generalization
The poster notes most of their monthly tasks are repetitive — exactly what agents should learn — yet current agent architectures can't, underscoring continual learning as a core gap.
More from coding & agent
- United Airlines to Ban Personal AI Agents From Booking Flights for Customers — Polymarket · 2026-11-12
- Agentic Search explained: a 4-search loop costs about 5x one search — and queries leak to more companies — adnan_hashmi · 2026-11-12
- AWS says thousands of autonomous AI agents wrote every line of AWS Emulator — SumitGup · 2026-11-12
- freeCodeCamp launches free 2-hour course on agentic coding with Codex — adnan_hashmi · 2026-11-12
- A criminally underrated resource comparing tool-calling reliability across models and providers — ricklamers · 2026-11-12
- Organizing 55k data points is easy: enterprise data prep for agents is the hard part — BenBajarin · 2026-11-12