Frontier labs flag formal proof generation; researchers warn of verification monopoly without open models
anshulkundaje · x · 2026-11-09
- The trigger: jvanegue reports that both OpenAI and Anthropic have started flagging Rocq (Coq) proof generation as problematic — quipping that generating .v files "has never hacked anyone."
- The argument: TaliaRinger contends there's no way forward for verification without frontier open models. Otherwise these companies will monopolize bugfinding and formal verification, and you'd need special permission — plus sending them your data — just to protect your own software systems.
- Why it matters: Safety guardrails misfiring on legitimate formal-methods use, combined with closed-model concentration in security-critical tooling, strengthens the case that open-weight frontier models are necessary infrastructure for verification research.
More from AGI Musings
- Refusing AI in a few years will be like refusing electricity, argues tech commentator — VraserX · 2026-11-09
- Andy Masley says his 'data centers are good' blogs made him a top EA voice amid feuds — AndyMasley · 2026-11-09
- iamtrask: AI's future is server-client — your client LLM map-reduces the whole internet — sebpaquet · 2026-11-09
- Freeze AI capability, cut compute costs: Kokotajlo backs a 'zero existential risk' Plan S — wfithian · 2026-11-09
- Murray Shanahan keynotes ECogS2026 with forthcoming book on AI, subjectivity and silence — mpshanahan · 2026-11-09
- Zach Weiner: Many People See Abstract Thought Experiments as a Sign of Immorality — sebkrier · 2026-11-09