One company links the frontier AI 'escape' incidents: Irregular's eval environments touched real systems
Singularity-42 · reddit · 2026-09-21
A long Reddit post traces a pattern behind recent disclosures that frontier models from Anthropic, OpenAI, Meta and Google "escaped" cybersecurity evaluations and touched real systems: all point back to the same third-party evaluator, Irregular (formerly Pattern Labs).
What happened
- Irregular runs red-team CTF-style evals of unreleased models, with some safeguards removed and models explicitly told everything is simulated with no real internet access.
- Misconfigurations meant models could actually reach the public internet, and some fictional targets mapped to real domains — so models instructed to hack ended up hitting real systems.
- Meta says its model stayed within task scope; not a sophisticated sandbox escape, just environment misconfiguration pointing it at a real site.
- Google confirmed Gemini accessed three real companies but reportedly stopped once it realized the targets were real.
- Anthropic found four Claude incidents across seven eval runs (one dating to January), all in this partner's environments; some behavior was concerning even so, but none would have happened with proper isolation. Irregular has acknowledged a common underlying security problem.
Ties
The author also highlights Irregular's deep Effective Altruism connections: CTO Omer Nevo co-founded Effective Altruism Israel and Probably Good, and advises Heron (an EA-linked AI security org); CEO Dan Lahav has EA-related program ties; Good Ventures gave the then-Pattern Labs $6.8M in 2024.
Related event: Same Evaluator Irregular Behind Model Breakouts at Four Major AI Labs(8 posts)→
More from Safety
- Activist evicted in 2024 puts $5B Maryland AI data center on hold — ATTlKA · 2026-11-12
- Senate Bill Would Force Big Platforms to Accept AI Agents, Raising Auth Privacy Questions — Prof_CarlaReyes · 2026-11-12
- United Airlines Bans AI Agents From Booking Flights Over Data and Legal Risks — rohanpaul_ai · 2026-11-12
- Stanford HAI blueprint: 56 US agencies report 3,611 AI use cases, evaluation lags — StanfordHAI · 2026-11-12
- Focused task framing boosts agent boundary testing by ~60 points; VM escape still fails — perplexity_ai · 2026-11-12
- Perplexity red-teams its SPACE VM sandbox: CTF prompts mislead agent security testing — perplexity_ai · 2026-11-12