ARC-AGI harness debate: OpenAI runs with provider adapter that preserves opaque reasoning across turns
giffmana · x · 2026-11-18
In the ARC-AGI harness discussion, mhmazur clarified how harnesses differ, responding to confusion raised by giffmana and others:
- Standard harness: models can carry notes between turns; long conversations get truncated
- Provider adapter harnesses: run with each provider's recommended API settings — for OpenAI that means preserving opaque reasoning between turns and auto-compacting long conversations
giffmana agreed the approach makes sense but noted the unexplained terminology caused trouble: it sounded like providers built special harnesses just for arc-agi (a terrible idea), which is where many of the bad replies and misunderstandings came from.
Related event: ARC-AGI Clarifies Provider-Adapted Harness Used in OpenAI's Benchmark Runs(2 posts)→
More from Models
- GPT-6.2 Astra and Sol roll out as testers push back on the hate wave — jeffintime · 2026-11-18
- GPT 6.2 Astra and Sol Take 4th and 5th on sudo L7 Coding Benchmark, Still Behind Latest Claude — echen · 2026-11-18
- Why did OpenAI ship a dud? Ex-engineer suspects cost-cut bet on RL backfired — doodlestein · 2026-11-18
- No single model matches Claude, says developer as Anthropic 'hits escape velocity' — _arohan_ · 2026-11-18
- Hands-on with GPT-6.2: Ignore the hate, it's a cheap computer-use workhorse — nickbaumann_ · 2026-11-18
- Claude leads Arena agent leaderboard with nearly 2x praise-to-complaint ratio — thursdai_pod · 2026-11-18