AI puzzles: a graded human-to-ASI problem ladder with checkable proofs — Grok fails first field test
jarekduda · reddit · 2026-10-25
Reddit user jarekduda, working with Claude and Sol, released "AI puzzles": a new format between fixed benchmarks and open-problem lists — a graded ladder of 9 puzzles from aperiodic order, from "human level" (pen and paper) up to "ASI level" (open).
Key design points:
- 5 solved rungs with programmatically verifiable exact answers for training and checking; 4 open rungs come with strong numerical evidence, known-failed routes, and machine-checkable success criteria
- One rule: only a checkable proof counts — floats are evidence, not proof
- Solvers of open rungs are credited by name, agent, and model in the next paper version
First field test with Grok: publicly asked, it claimed a proof for an open rung that was only a numerical estimate, then retracted it; it got the human-level rung wrong at first, fixed it after one hint, and proved the expert rung on the first try. The dataset is on HuggingFace, and the top rung is already being tested by an expert on MathOverflow.
Related event: New tiered verifiable math puzzles debut, Grok fails first test(2 posts)→
More from AGI Musings
- Indian Premier Institute Students Still Build Outdated Projects, Unaware of AI Disruption — HarveenChadha · 2026-10-25
- Claude-made video pushes back on FOOM doomerism: predicted AI takeoff never happened — FinanceYF5 · 2026-10-25
- Noah Smith: AI cyberattack wave may end the internet as a public good — skdh · 2026-10-25
- Heavy AI User Has Claude Build Weekly Infographics to Fix Her Own Memory Lapses — LadyAshBorg · 2026-10-25
- "We were burning whales": a viral reflection on standing at the singularity's edge — geoffwolfe · 2026-10-25
- 'A maximally truth-seeking AI might actually be a misaligned AI' — eigenron · 2026-10-25