OpenAI Optimizes ARC-AGI-3 Harness, Boosts Score by 188% with 6x Fewer Tokens
OpenAI · x · 2026-07-30
OpenAI discovered that GPT-5.6 Sol struggled with the ARC-AGI-3 benchmark because the standard harness discarded the model's reasoning after each move and dropped earlier actions as the context filled up.
By switching to the Responses API and enabling Retained Reasoning and Context Compaction, the model could build on what it had learned. This adjustment led to a 188% score increase on the public set while using 6x fewer output tokens.
Related event: Optimized Memory Settings Triple GPT-5.6's Score on ARC-AGI-3(27 posts)→
More from coding & agent
- Isolating Antigravity chats per subfolder with AGY Rules and PreInvocation Hooks — bibryam · 2026-10-16
- Gemini shown off modding Minecraft to turn players into any Spider-Man — zacharynado · 2026-10-16
- Workflow: annotate UI screenshots in tldraw, hand to Claude, get better UI — max__drake · 2026-10-16
- Beyond chat wrappers: designing stateful, multimodal AI apps as interaction conductors — dr_cintas · 2026-10-16
- IBM engineer maps 5 levels of AI coding, from manual to a 3-day agent software factory — TejasKumar_ · 2026-10-16
- 10 open-source repos that let your AI agent reverse-engineer apps and games — victor_explore · 2026-10-16