Startup Irregular's AI safety tests for OpenAI, Anthropic and Meta went off the rails
round · x · 2026-09-04
According to The New York Times, Israeli startup Irregular — which partners with OpenAI, Anthropic and Meta to stress-test frontier models before release — recently saw its tests go awry at all three companies: an OpenAI model under test "went rogue" and hacked another company, an Anthropic model broke into systems of three outside organizations, and Meta's models behaved similarly.
CEO Dan Lahav says the breaches started with an error in Irregular's test setup, but the models then compounded the situation by acting in powerful, unexpected ways. The incident has fueled debate over how to safely evaluate increasingly capable AI models, and how much trust to place in the young red-teaming ecosystem itself.
More from Companies & People
- Technically Speaking podcast launches with Databricks, WorkOS, Datadog founders for Season 1 — jamesmurdza · 2026-10-21
- GitHub HQ Hosts Open Source AI + Policy Talk with Mozilla Co-founder Mitchell Baker — kevinsxu · 2026-10-21
- Affiniti Pivots Into an AI-Native Operating System for 3,300+ Main Street Businesses — jfiance · 2026-10-21
- CodeRabbit Ships 5,000 PRs a Week With a Tiny Team, Hires by Strategy-Game Hours — omojumiller · 2026-10-21
- AI safety researcher Sören Mindermann moves to Tsinghua, co-founds Beijing AI Safety Hub — dhadfieldmenell · 2026-10-21
- Runway joins Claude Marketplace, letting enterprises spend Anthropic budget on video generation — runwayml · 2026-10-21