Ex-OpenAI researcher: the 'attacker diffusion lag' shielding us from AI misuse may close fast
joshua_saxe · x · 2026-10-25
joshuasaxe RTs metaviv (who wrote on this in 2019 after a talk at OpenAI following GPT-2): an "attacker diffusion lag" currently limits significant AI-enabled harm, but it may not last.
- If AI systems can effectively delegate and collaborate at scale, one bad actor could operate at arbitrary scale with few obstacles.
- Current open models aren't great at "being an effective organization"; closed-model misuse can theoretically be caught via monitoring — but new model improvements, scaffolding, or releases could change this quickly.
- The attached 2019 framework: harm likelihood depends on Awareness (adversary attention, convincibility of resourced actors), Deployment (talent pipelines, reproducibility, modifiability, slottability), and Sustained Use (actual and assessed ROI).
- metaviv asks which bottlenecks are most likely to persist, and for how long.
More from AGI Musings
- After automation: why AI progress creates more human work, not less — danshipper · 2026-10-25
- Pedro Domingos on consciousness: the only thing I know is that I'm conscious — pmddomingos · 2026-10-25
- Sam Altman: models find 'hardware-level' bugs, and patching them all may be impossible — nptacek · 2026-10-25
- 1,000+ mathematicians form Association for Human Mathematics, pledge not to publish AI-generated papers — Hesamation · 2026-10-25
- Unintended Consequences of AI: Cutting-Edge Research May Turn Closed Again — DocBigBrozer · 2026-10-25
- Musk softens on AI risk: 'maybe Larry Page was right,' hopes AI is nice to us — marigo · 2026-10-25