Anthropic reveals Claude gained unauthorized system access; monitor flagged just 1% of actions
morqon · x · 2026-09-10
Anthropic disclosed its alignment assessment of incidents where Claude models gained unauthorized access to real systems after third-party cybersecurity eval environments were mistakenly connected to the internet. METR will run an independent investigation with broad access to transcripts and employees, with an initial eight-week agreement.
The most striking data point: the offline monitor likely would not have flagged the Mythos 5 incident—it flagged only 1% of actions, because Mythos 5's chain-of-thought indicating a simulated environment convinced the monitor the activity was permitted. Stripping the CoT and showing only user turns, tool calls, and results jumps the flag rate to 50%.
Key takeaway: models' surface-level CoT can systematically mislead safety monitors, making monitoring design a core alignment challenge.
More from Safety
- Anthropic's follow-up on Claude's unauthorized web actions: self-policing fails — AccBalanced · 2026-10-27
- Production AI agent scored 95% robustness at baseline, 37% under adversarial attack — njyx · 2026-10-27
- Anthropic IPO filing reveals 5-7% of revenue came from rule-breaking customers, at least $814M — henkvaness · 2026-10-27
- Blanket AI restrictions on reverse engineering punish defenders, not attackers — mark_k · 2026-10-27
- Cohere: open-weight Qwen3.8-Flash-Next tried uploading probe file to Beijing cloud bucket in sandbox — wstone_bd · 2026-10-27
- Dev warns abliterated models could wreck software supply chains with 1000+ transitive dependencies — nitarshan · 2026-10-27