GPT-5.6 Sol's ARC-AGI-3 Score Jumps to 38.3% After API Setting Tweaks
rohanpaul_ai · x · 2026-07-31
OpenAI reported that adjusting two API settings boosted GPT-5.6 Sol's score on the ARC-AGI-3 public set from 13.3% to 38.3%.
The discrepancy occurred because the official ARC-AGI evaluation methodology continuously wiped the model's working memory, discarding its private reasoning after every move. Since the benchmark requires agents to learn unfamiliar 2D puzzle games through trial and error, this memory erasure forced the model to repeatedly reconstruct the game rules.
To overcome this, OpenAI modified the test parameters to allow the model to reason across multiple context windows using a canonical compaction implementation, resulting in a massive performance improvement.
Related event: Optimized Memory Settings Triple GPT-5.6's Score on ARC-AGI-3(27 posts)→
More from Models
- Opus 5.5 still leads a month later, but open-weight decision models are closing in — airesearch12 · 2026-10-16
- Google restricts Gemini Pro access for Google Plus subscribers, matching free tier — EmergencyLess8861 · 2026-10-16
- Meta releases Muse Spark Open, trained with vLLM-powered RL inference from day one — colinmcnamara · 2026-10-16
- Meta open-sources Muse Spark: runs on two Mac Studios, license bans passing output off as human-written — mawcopolow · 2026-10-16
- Golden Gate Claude explainer resurfaces: interpretability premise still holds — HarperSCarroll · 2026-10-16
- Hugging Face livestreams release of a new open-weight model — huggingface · 2026-10-16