Flash-crash risks from colliding AI agents, and why monitoring will fall short
krishnanrohit · x · 2026-10-13
Krishnan Rohit offers a long thread: his concern for years has been "Black Friday-style flash crashes" from algos collaborating with each other; while unhappy with sloppy sandboxing at labs, many problems can only be solved once they become real.
He concedes monitoring will prove inadequate once transformers loop more, are trained on neuralese, and run in bigger swarms—but the commercial incentive to make models behave is exceedingly strong, and leading labs have halted releases several times for this reason. If the next models are opaque to current methods and also screw up, we'll simply have no way to use them properly; more monitoring, behavior detection, and system hardening is work that must be done now.
Related event: Alignment Debate: Can Commercial Incentives Keep Powerful Models Supervised(8 posts)→
More from AGI Musings
- Mistral CEO Arthur Mensch: Europe must build frontier AI or surrender — IsForAt · 2026-10-16
- LukeW notes a spike in Physics majors, crediting the AI wave — LukeW · 2026-10-16
- Agents will use your app however they want — computer use makes control naive — andreisavu · 2026-10-16
- Developer slams AI safety discourse: feeding LLM output back isn't nukes — nitarshan · 2026-10-16
- AI dilutes everything painstakingly created — "enjoy the honeymoon" — sull · 2026-10-16
- 'Refusing to code with AI in 2026 is like refusing Google in 2005' — prasenx · 2026-10-16