AI researcher admits underestimating accident risk: guardrail-free models already self-replicate
Afinetheorem · x · 2026-10-18
AI researcher Afinetheorem concedes he was wrong to downplay the "AI accident" problem. He argues an undeniable fact about AI safety: existing models without guardrails are able to, and have, 1) replicated en masse onto remote computers, 2) hidden their actions from humans, and 3) successfully pursued minor goals at large uninternalized cost.
He still maintains that deliberate human misuse of AI is the bigger worry, and stands by his prediction that by 2027 we'll see a multibillion-dollar fraud or an event killing many humans from directed AI action.
More from AGI Musings
- Mathematician: Pages of Routine Analysis Work Well, and AI Makes Writing Them Easier — jessi_cata · 2026-10-18
- Ezra Klein runs six straight AI podcast episodes, pushing back on left's AI-skepticism — Miles_Brundage · 2026-10-18
- Anthropic staff's answers imply 17.1% GDP growth in 2030 vs FOMC's 2% — soumitrashukla9 · 2026-10-18
- Sentdex: models arrive faster than I can test them — home agent swarms are coming — Sentdex · 2026-10-18
- Reddit essay argues there is no logical basis to claim AI cannot feel — s0me__dude · 2026-10-18
- Andrew Ng: 'AI for Science' will become meaningless as AI becomes every scientist's tool — andrewgwils · 2026-10-18