Security take: swarms of AI attack agents are coming for code nobody fully understands
mfckr_eth · x · 2026-10-23
The author argues from a cybersecurity angle that AI acceleration makes existing software dangerously fragile:
- Human error is the core problem: most engineers don't truly understand why their code works ("if it works, don't touch it"), and swarms of thousands of highly capable, fast AI attack agents will exploit that.
- Possible regression: if digital systems become unusable due to automation-driven attacks, society may fall back to unhackable brick-and-mortar systems or simpler monolithic software.
- Offense-defense asymmetry: far more people are trying to break into systems than to defend them, with wildly misaligned incentives.
More from AGI Musings
- Anil Seth: Consciousness Is a Property of Life, Not Computation — Building Conscious AI Is a Bad Idea — anilkseth · 2026-10-23
- Pedro Domingos predicts 2027 will see the most security vulnerabilities fixed in history — pmddomingos · 2026-10-23
- AI+Lean theorem proving loses to Bitter Lesson: OpenAI math gains come from non-formalized reasoning — Singularitarian · 2026-10-23
- Beff Jezos calls for a 'cyber immune system' of constantly adaptive agents — beffjezos · 2026-10-23
- Pedro Domingos argues RL solves nothing that better generalization can't — pmddomingos · 2026-10-23
- Academic Writing in the AI Age: Do You Write for Humans or for LLM Reviewers? — JeremyNguyenPhD · 2026-10-23