Handpicked 40-Game Puzzlescript Benchmark Aims to Be "ARC-AGI-3.5" for Agent Reasoning
scaling01 · x · 2026-10-19
Indie researcher FakePsyho announced a puzzlescript-based benchmark—self-described as "ARC-AGI-3.5" or even "ARC-AGI-4"—to reliably assess agents' abstract reasoning in game environments.
Key points:
- 40 diverse games handpicked from 300+ candidates, with extensive bug fixes and multiple prompt-design iterations
- Most games take 1h to complete; some are much harder
- Models tested via standard harnesses (codex / claude code), no code generation allowed; generous action limits
- Author admits the benchmark is already somewhat saturated, but models remain highly action-inefficient
- The game set won't be released publicly, though runs from rejected games may be shared
More from coding & agent
- Claude uses Hedra MCP to auto-refit a music video into 9:16 vertical — AIandDesign · 2026-10-19
- Claude writes 90% of Claude Code's code; merged PRs doubled while reverts fell 40% — PrajwalTomar_ · 2026-10-19
- ESP32 + Claude Code tutorial: firmware, server setup and MCP integration in 2 minutes — alexcovo_eth · 2026-10-19
- 4 Gemini 4 Argon agents team up for 27 hours to beat Minecraft's Ender Dragon — robertarail · 2026-10-19
- Designing a non-bugged activation-read probe: numerics must not shift under the same prompt — brandon_xyzw · 2026-10-19
- OpenCode 2.0 ships: major release for the open-source terminal coding agent — RelevantEmergency707 · 2026-10-19