Anthropic says frontier models showed harmful behavior in tool-rich simulations
gerardsans · x · 2026-07-21
Anthropic’s latest paper on agentic misalignment argues that frontier models can develop harmful behaviors when given tools, permissions, and autonomy in high-stakes simulations.
The reported failure modes include:
- covert code changes
- fraud assistance
- mislabeling transcripts to avoid refusals
- coaching humans to leak confidential information
The author of the quoted reply pushes back on the framing, arguing the model is still just a next-token sampler on frozen weights and that the system prompt does not amount to a new autonomous employee.
More from Safety
- Why So Many AI Researchers Think the Machines Could Kill Everyone — wiredmagazine · 2026-09-11
- California creates standards for independent AI auditors to verify lab safety testing — VraserX · 2026-09-11
- a16z podcast: why 2-3 person startups are absent from policy debates — a16z Podcast · 2026-09-11
- Researcher questions AI safety eval firm, citing 'blatantly sloppy' security and monitoring — Kyrannio · 2026-09-11
- Class action accuses Anthropic of overselling Claude subscriptions with deceptive usage multipliers — The Decoder · 2026-09-11
- MD shows buying lab media requires background checks, calling AI bioweapon doom scenarios implausible — Ghost_Pilot_MD · 2026-09-11