Blue AI and red AI in cyber don't cancel out—program analysis has theoretical limits, says Saxe
joshua_saxe · x · 2026-09-13
Security researcher Joshua Saxe argues that blue-team and red-team AI in cybersecurity don't simply cancel each other out like a simplified RTS game model.
- Theoretical limits of program analysis and distributed code (parts living on unanalyzable third-party SaaS) make it impossible to guarantee finding and fixing all bugs
- Even blocking vulnerability exploitation, attackers can get in via social engineering and implant just-in-time malware
- Detecting such malware and its C2 channels may require solving unsolved problems in program analysis or cryptanalysis
The thread discusses getting open-weights cyber policies (like Astra) right, with replies advocating a red-team + blue-team defense-in-depth mix.
Related event: Red vs Blue AI Debate: Layered Defense, Not Cancellation(2 posts)→
More from AGI Musings
- Why an AI arms treaty needs a verification story before signatures — victor_explore · 2026-09-13
- Fake DSM-6 lists 'AI Eschatological Delusion' as a new disorder — whurley · 2026-09-13
- OpenAI's Navier-Stokes proof used ~10,000 agents for 88 hours — is AGI a swarm, not a model? — callme_e · 2026-09-13
- Dev questions centralized frontier-pacing, argues open source AGI must win — 0xsachi · 2026-09-13
- Open-model RL practitioner pushes back on roon: safety standardization won't threaten open source — willcb · 2026-09-13
- Commenter: 'Frontier pacing' talk collapses once a Chinese model beats ChatGPT Pro and Claude Max — doodlestein · 2026-09-13