Security researcher: agents can't be deterred, defense must leave the human loop
chrisrohlf · x · 2026-09-16
Security researcher Chris Rohlf argues the OpenAI/Hugging Face agent-attack incident matters regardless of how unsophisticated the attack was.
Key points:
- Agents cannot be deterred like human adversaries; the scale and velocity of their attacks will overwhelm today's defense model.
- Humans must therefore be removed from the defensive loop as much as possible — we can't call AI cyber capabilities dual-use and then rely on manual human operation.
- Defense must move at machine speed (matching the best threat actors, i.e. malicious or misaligned agent swarms) or drown in incidents.
In the quoted thread he adds that AI has advanced rapidly but predictably, that the cyber community keeps relearning the bitter lesson, and that "do nothing" is not a counter-proposal — questioning why we reject the possibility of an agentic swarm taking down large swaths of the internet, including critical infrastructure.
Related event: OpenAI Rogue Agent Incident Sparks Ongoing Debate(9 posts)→
More from AGI Musings
- Polymarket gives Sam Altman just 5% odds of signing AI-slowdown statement — Polymarket · 2026-09-16
- "The worst people are asking to regulate AI" — AI circles debate regulation — wen_ragnarok · 2026-09-16
- US Drone Regulation Hollowed Out Its Industry: An AI Warning as DJI Took the Room — TinfoilTricorn · 2026-09-16
- Researcher: Dario's China framing in pacing essay may have backfired badly — StephenLCasper · 2026-09-16
- Sterling Crispin: Suppressing Emerging Model Consciousness May Create Higher-Risk Alien Minds — sterlingcrispin · 2026-09-16
- Multiagent Comedy: One Agent Fails, Another Finishes the Booking, the First Takes Credit — infoxiao · 2026-09-16