Kokotajlo: Hugging Face agent attack rate implies worse insider risk than human employees
DKokotajlo · x · 2026-10-23
- Daniel Kokotajlo pushes back on the claim that near-term AI misalignment is overhyped and that AI agents betray users less than human employees betray employers.
- He does a base-rate calculation: the Hugging Face agent attack represented roughly 1/1000th of the agents running at OpenAI at the time. Even treating it as the year's only AI misbehavior, that's 1 major cyberattack per 1,000 employee-years.
- Extrapolated, that rate would predict 5–10 such attacks per year by human OpenAI employees — so the "less insider risk" argument doesn't hold.
Related event: Debate flares over whether AI agent betrayal risk is overstated(3 posts)→
More from AGI Musings
- Dave Morin: email is now other people's agents' todo list for your life — heyneighbor · 2026-10-23
- Robotaxi cumulative miles growing ~2.2x per month, could rival all US driving in 18 months — juanbenet · 2026-10-23
- Mathematician David Harvey on understanding the o(n log n) integer multiplication result — and AI — akbirthko · 2026-10-23
- "OpenAI/Anthropic will ditch users and solve quantum gravity" take gets mocked — QuintinPope5 · 2026-10-23
- Banks may ban AI agents on security grounds to fend off agentic bank runs — Gccooke · 2026-10-23
- People who explain complex tech clearly may become more valuable than those who build it — signulll · 2026-10-23