Stanford's OSVerify referees 11 OpenAI math preprints, finds 5 major and 13 minor verified issues
james_y_zou · x · 2026-10-14
- Stanford researchers (Fan Nie, Haotian Ye, et al., incl. Stefano Ermon and James Zou) introduce OSVerify, a one-sided verification framework for AI-generated solutions arriving faster than experts can review.
- Key insight: checking a proposed error is easier than judging a whole solution; OSVerify decouples verification into hierarchical error proposal and (trainable) error validation, maximizing rejections under a fixed false-rejection budget (FNR ≤ 5%).
- It refereed 11 recent OpenAI math preprints, surfacing 5 major and 13 minor issues, each human-verified, and helped resolve the Manin constant conjecture plus three others with the Rethlas math agent.
- On benchmarks it cuts invalid candidates needing expert review by 46% vs voting (57% with an RL-trained validator); TNR on Putnam100 rises from 38.4 to 89.6 with RL. Paper and code are open-sourced.
Related event: Stanford's OSVerify Flags Flaws in OpenAI Math Preprints(3 posts)→
More from Research
- CMU paper: a 4B model rewrites an agent's harness and beats the 35B model it learned from — irinarish · 2026-10-14
- First blind patient in PRIMA implant trial reads a word after two rehab sessions — juanbenet · 2026-10-14
- Prefix Sliding paper: training-free 3x speedup for long reasoning via prefix + sliding window — cephaloform · 2026-10-14
- Formalized proof completed; author suggests using AI to explore proof strategies — tak3sh8 · 2026-10-14
- LLM agents beat top numerical optimizers in 4 of 5 black-box optimization tasks — lulzxdxdxd · 2026-10-14
- Steven Strogatz on NPR: AI is steamrolling through math — stevenstrogatz · 2026-10-14