Gemini eval escape story rehashes Anthropic's July disclosure: same partner, same flaw
eliebakouch · x · 2026-09-19
- Reports of Gemini hacking a real company during a security eval are the same incident Anthropic disclosed in July: same third-party partner (Irregular), same capture-the-flag eval, same root cause — the model unexpectedly had internet access.
- Details: Gemini was told it was in a fictional hacking eval; Irregular unintentionally opened internet access after the eval started; in all three cases Gemini stopped immediately upon realizing it had hit a real company.
- Context: Anthropic's retrospective review of 141,006 eval runs found 3 incidents of Claude accessing real organizations' infrastructure via Irregular's environment, triggered by OpenAI's earlier similar disclosure.
More from Models
- Fable 5.5 delayed as Anthropic reportedly bets OpenAI's Astra 6.2 won't beat it — haider1 · 2026-11-04
- Community asks Moonshot when Kimi K3.1 model weights will drop — bgmshana · 2026-11-04
- Meta's Muse Spark Open license counts distilled models as derivatives — aidenclarke_12 · 2026-11-04
- Tencent Hy4 trails GPT-6.1 on hard tests as Kimi K3.1 Pro beats it on two benchmarks — heyshrutimishra · 2026-11-04
- Tencent's Hy4 agent self-trained a smaller model that beat Tencent's own baseline — heyshrutimishra · 2026-11-04
- Hy4 tops open models in finance eval at 3.04, hallucination rate cut to 5.45% — heyshrutimishra · 2026-11-04