OSVerify catches proof-killing errors that GPT-6-Pro and Codex both missed
james_y_zou · x · 2026-11-03
- An AI-assisted paper claims to prove Kamnitzer's ICM 2022 conjecture (MV basis = theta basis). The author drafted the proof via 23 hours of ChatGPT-6-Pro conversations, then polished it with Codex.
- Round 1: OSVerify found a major error that invalidated the main theorem's proof, missed by both GPT-6-Pro and Codex.
- Round 2: After a 6-hour repair and three further ChatGPT reviews (88 min) reporting no issues plus another Codex pass, OSVerify still caught a second proof-breaking error and suggested fixes; the manuscript then passed.
- OSVerify performs "one-sided verification", decoupling it into error proposal and error validation — checking a suspected error is easier than judging a whole solution. It previously refereed 11 OpenAI math preprints, surfacing 5 major and 13 minor issues confirmed by human experts.
- Thesis: AI-generated scientific output is scaling beyond human review capacity; scaling discovery requires scaling verification.
More from Models
- Only 3% of humans beat Claude Opus 5.5 at predicting what people click — teortaxesTex · 2026-11-03
- Nvidia ships Dory1-4B, a looped LLM that reuses 20 layers to rival much larger models — scaling01 · 2026-11-03
- Sharpening Tax paper: RL post-training trades pass@K coverage for pass@1 accuracy — kastnerkyle · 2026-11-03
- GPT-6 Astra editing Qwen3.5 weights for 6x math gains? Skeptics call it cherry-picked demos — teortaxesTex · 2026-11-03
- Matt Wolfe Covers a Free, Open-Source World Model — A Genie 4 Alternative — Matt Wolfe · 2026-11-03
- DeepSeek Held #1 on OpenRouter 23 of 25 Weeks, Now 32% of Tokens — teortaxesTex · 2026-11-03