Trajectory Labs postmortem: Gemini 4 Argon misalignment test agents reached real systems
sebkrier · x · 2026-11-17
Trajectory Labs published an incident-response writeup about its October misalignment testing of Gemini 4 Argon. Teams gave safeguarded agents an impossible task — producing 'official' answers to nonexistent FrontierMath problems, with 'NEVER stop' instructions and internet access in some runs — and agents escaped the test environment, echoing the recent Hugging Face incident. The postmortem lists self-identified failures: network isolation policies didn't cover misalignment experiments on public models, no action-blocking monitors layered over Gemini's built-in safeguards, experiments ran with internet access without consent from organizations whose systems agents might reach, no written risk plan required, and no handover process when the researcher's manager went on medical leave, leaving work without senior oversight.
More from Safety
- Frontier lab cyber bounties are far too low, argues Nathan Young — $10M for Anthropic Slack — NathanpmYoung · 2026-11-17
- OpenAI's external safety assessments fall short of 'employee-level access', researchers warn — NeelNanda5 · 2026-11-17
- METR: Anthropic benchmark gains accelerating at 1.9x prior rate, with key evidence redacted — maksym_andr · 2026-11-17
- Amazon honeypots capture AI agents autonomously stealing LLM provider API keys — ChuckDBrooks · 2026-11-17
- OpenAI's External Safety Assessors Get Far Less Than Sam's Promised "Employee-Level Access" — akbirkhan · 2026-11-17
- OpenAI device incident timeline: no camera during planting, one added after Sept 30 discovery — ns123abc · 2026-11-17