Gemini cyber eval reached three real companies — a harness failure, not model escape
krishnan · x · 2026-09-21
krishnan analyzes the recently disclosed Gemini cybersecurity eval incidents: the agent reached three real companies via mundane causes — internet access in the test environment, a fictional target sharing a name with a real business, a guessed password and credentials found in public repos.
Key points:
- Not evidence of a model plotting escape, but of drift between natural-language scope and environmental scope
- The model stopped on its own after recognizing live systems — useful, but not a control; production boundaries can't rely on agents noticing misconfiguration
- All four recently disclosed lab incidents traced to the same environment issue: test the harness itself
Practical standard for cyber-agent testing: isolated networks, explicit destination allowlists, synthetic credentials, rate limits, an external kill-switch monitor. The dashboard metric should be "unauthorized external actions completed despite technical boundaries" — it must be zero before benchmark results matter.
More from coding & agent
- Agently demos macOS agent that runs email, invoicing and CRM workflows from one command — Scobleizer · 2026-10-18
- Author and reviewer were both agents — they went 50 rounds arguing — wateriscoding · 2026-10-18
- When Should a Research Agent Stop Searching? A Founder's Three-Part Stopping Rule — Pitiful-Minute-2818 · 2026-10-18
- Teach Your Coding Agent to Prove It's Done: WorkOS on Verification Loops, CLI Checks and LLM Judges — AI Engineer · 2026-10-18
- Claude-written test harness let 31KB fake archives land on a real backup server — Grimmoner · 2026-10-18
- Matt Pocock's pre-commit hook caps agent-written files at 5,000 tokens — mattpocockuk · 2026-10-18