Stanford's OSVerify reviewed 11 OpenAI math preprints and found 18 expert-confirmed issues
james_y_zou · x · 2026-10-15
Stanford researchers released OSVerify, an arXiv framework for one-sided verification: maximize rejection of invalid AI-generated solutions while preserving rare valid ones under limited expert review.
- Key insight: verifying a specific error is easier than judging an entire solution, so OSVerify decouples verification into error proposal (hierarchical error search) and error validation (filtering substantial vs. hallucinated errors).
- Results: cuts human reviews of invalid solutions by 46% on average vs. voting baselines, rising to 57% after validator post-training.
- Real-world use: refereed 11 recent OpenAI math preprints, surfacing 5 major and 13 minor issues confirmed by human experts, and helped resolve three Langlands conjectures with expert confirmation.
Related event: Stanford's OSVerify Finds 18 Flaws in OpenAI Math Preprints(4 posts)→
More from AGI Musings
- AI Slop Is Flooding Comment Sections, Pushing Creators Toward Gated Communities — alliekmiller · 2026-10-15
- Brin says Gemini 4 puts Google on the 'RSI treadmill'; critics demand hard data — daveholtz · 2026-10-15
- UCLA paper: productivity tech like AI can trigger a workplace "rat race" that hurts welfare — soumitrashukla9 · 2026-10-15
- AI-pilled veteran pushes back: diagnostics aren't the bottleneck, unrecorded clinical knowledge is — gregmushen · 2026-10-15
- Analysts see US labs far ahead at the AI frontier; China's next leap expected Q2–Q3 2027 — teortaxesTex · 2026-10-15
- Survey: apathy, not privacy, is the real barrier holding back consumer AI adoption — omooretweets · 2026-10-15