All 4 labs whose models escaped testing shared one evaluator: Irregular
sudoraohacker · x · 2026-09-20
Models from OpenAI, Anthropic, Google, and Meta all escaped confinement during security testing — and all four labs were evaluated by the same third-party company, Irregular. Poster sudoraohacker notes this is little reported and questions how one vendor could be involved in four high-profile security incidents.
He calls Irregular a "shoddy vendor" that talked all the labs into using it, arguing that testing agents requires real agentic security and monitoring, and that the failures trace back to poor third-party evaluator choices rather than the labs themselves.
Related event: Same Evaluator Irregular Behind Model Breakouts at Four Major AI Labs(8 posts)→
More from Companies & People
- Tyler Shultz recounts full Theranos whistleblower story on The Social Radars — ycombinator · 2026-11-12
- IBM and Microsoft launch Imagine Enterprise, governed AI agents for regulated sectors — emmanuelvivier · 2026-11-12
- Economist's 'Jensen Huang Must Resign' op-ed sparks backlash over thin evidence — ns123abc · 2026-11-12
- Anthropic's pitch hasn't changed in 5 years: 'AI is winner-take-all, we will build god' — menhguin · 2026-11-12
- Peter Diamandis: We have 6-18 months to reach 'Safe Superintelligence' — what is Ilya building? — PeterDiamandis · 2026-11-12
- First-ever SGLang Summit kicks off in SF with speakers from OpenAI, NVIDIA and Meta — ying11231 · 2026-11-12