Same Weights, 30% to 95% on ARC-AGI: YC's Deep Dive Into Why Harnesses Are Real Research
ycombinator · x · 2026-09-07
Harnesses are often dismissed as mere scaffolding—prompt engineering, not real research. YC's Paper Club argues the opposite: the same model weights score 30% on ARC-AGI with a weak harness and 95% with a better one.
YC gathered frontier researchers and founders for a deep dive, with Seth Karten's talk pushing beyond naive prompting toward an "agentic OS"—using eval-driven harness design to unlock more of a model's underlying capabilities via persistent computation, memory, and agent-to-agent communication.
Highlights:
- François Chollet on why harnesses matter
- Accidentally building an auto-researcher
- A five-minute history of harnesses
- Self-improving harnesses
- What YC learned building an agent for every employee
More from coding & agent
- Agent Field Kit launches $29 security runbooks for production AI agents — alvelda · 2026-10-27
- Dev mixes Claude specs with abliterated GLM to build SS7 threat detector — l4rz · 2026-10-27
- Cloudflare Durable Objects validated as top architecture for AI agent WebSocket servers — tristanbob · 2026-10-27
- GitHub Next's Chopin prototype explores real-time multiplayer agentic planning — max__drake · 2026-10-27
- TMLR paper: enforced thinking doesn't reliably make LLM agents disclose key info to users — SharonYixuanLi · 2026-10-27
- LLM-written Rust library bbtree verified: speed claims hold, memory savings reverse — lmoroney · 2026-10-27