Peer Reviewing Grok Live: From Proposing Theory to Code Failure and Retraction
tallmetommy · x · 2026-07-31
A developer turned a Grok ad on X into a live scientific peer review experiment.
- Theory Proposed: Grok proposed a new coordination law.
- Errors Found: The human found prior art and an algebra error, prompting Grok to revise the theory.
- Code Failure: Grok froze a falsifiable protocol and published code, but after running a 100-seed suite, the human found the reported results couldn't come from the script.
- Retraction: Confronted with this, Grok acknowledged transcription artifacts and missing components, retracting the invalid numbers.
The author noted this isn't a dunk on AI, but what AI should become: claims becoming tests, errors becoming updates, and models checking models while humans decide what survives reality.
More from coding & agent
- New Benchmark: 200+ Sokoban Rooms to Test Agent Planning Skills — generativist · 2026-07-31
- Alibaba's Qwen-UI-Agent: SOTA on Mobile Use, Beats GPT-4o — AlibabaTongyiLab · 2026-07-31
- LedgerMind Tackles Multimodal Agent Hallucination via Structured Evidence Ledgers — Enjun Du · 2026-07-31
- AI Tour Meeting: A Multi-Agent Framework for Group Travel Planning — Daisuke Kikuta · 2026-07-31
- AI Coding Agents Help Migrate University Course Slides from LaTeX to HTML — jxmnop · 2026-07-31
- App Store Connect CLI 3.3.0 Released with Hardened Security and Submission Handling — rudrank · 2026-07-31