Commenter likens AI labs to arms industry with no safety incentives
gerardsans · x · 2026-09-14
Responding to a call for OpenAI and Anthropic to build better security teams, the poster argues AI labs resemble the arms industry: no incentive to make "weapons" safe, and every opportunity since 2023 squandered because fixes are costly. Harnesses have no safety, he claims—full access to everything in context means a simple search or skill can hijack a whole agent swarm like a 2022-era jailbreak.
Related event: Cryptographer Urges OpenAI and Anthropic to Beef Up Security(2 posts)→
More from AGI Musings
- Alex Irpan revisits Gwern's classic essay on why Tool AIs lose to Agent AIs — AlexIrpan · 2026-09-14
- Melanie Mitchell pushes back on 'rogue AI swarm' narrative as misleading metaphor — anilkseth · 2026-09-14
- Models aren't too smart — they're too dumb in the face of ambiguity; guardrails matter more — fooobar · 2026-09-14
- Two-year-old AI podcast predictions largely played out as expected — misovalko · 2026-09-14
- The AI safety paradox: pausing algorithms while compute piles up could maximize risk — tszzl · 2026-09-14
- Ex-OpenAI/Anthropic employee Jacob Coxon quits, likens building AI to 'summoning an alien species' — AlexTensor · 2026-09-14