Continual-ARC: lifelong ARC-AGI stream finds agent harness persistence rarely beats simple caching
unifyai · hf · 2026-11-16
Unify AI proposes Continual-ARC, a lifelong stream built from the three ARC-AGI editions where tasks return after dormant gaps (testing memorization) and new tasks keep arriving (testing generalization). Learners get a per-instance feedback budget to buy demonstrations, submit answers, or act inside the harness, scoring memory- and weight-based systems on one scale. Frozen-weight experiments on Qwen3.5-9B and GLM-5.3 Flash show harness persistence rarely improves memorization, most harness methods underperform simple caching, and weight-based learning generalizes better.
More from coding & agent
- WebArena-Pro: a 300-task multimodal benchmark across 20 self-hosted apps for web agents — McGill-NLP · 2026-11-16
- Filmworkbench v0.1.2: one-prompt semi-automated short films with Claude and ComfyUI — Eastern-Ad9515 · 2026-11-16
- Stop prompting LLMs for JSON: PowerShell module Jev routes with decision models — dfinke · 2026-11-16
- llama.cpp offloads CI caches and runners to Hugging Face to survive 35 PRs/day — ggerganov · 2026-11-16
- I killed my 30-minute weekly sync with a decision log my agents read before every run — Humaux · 2026-11-16
- A video editor tested free MCP-connected AI editing tools: invideo, ChatCut, FableCut — lol_you_again · 2026-11-16