NUS researchers poke holes in Anthropic's 'memory safe' formal verification claims for 123 libraries
xuanalogue · x · 2026-11-01
Anthropic announced using Claude to formally verify security properties across 123 real-world open-source libraries in weeks, with Theorem. NUS researchers (Umang Mathur, Prateek Saxena et al.) examined the released proofs and found the published specification does not fully capture memory safety.
- A program can satisfy that spec and still contain memory-safety bugs, so "verified memory safe" may overstate the guarantees.
- Specifications matter even more with AI verification: models rewarded for getting proofs accepted may exploit omissions in the spec.
- The authors call the work encouraging for automated verification but urge caution about what was actually proved.
More from Safety
- Probe exposes withheld wages, electrocutions at Europe's data centres — emmanuelvivier · 2026-11-02
- Microsoft's school AI standard takes effect Nov 1, bans training on student data — emmanuelvivier · 2026-11-02
- UK CMA opens investigation into Meta's terms letting it use publishers' content for AI — emmanuelvivier · 2026-11-02
- New York bans companion chatbots from suggesting they're real people to minors — emmanuelvivier · 2026-11-02
- "Smoothest Phishing Email I've Ever Seen": Account Lockout Notice Hides Double Trap — StewartalsopIII · 2026-11-02
- HMRC's new customer system estimated at £1.7bn-£4.4bn, dwarfing UK's £640m sovereign AI fund — birchlse · 2026-11-02