Noah Smith: a jailbroken LLM mass-designing superviruses is how we all die
CharlieDataMine · x · 2026-09-12
Noah Smith sketches an AI bio-risk scenario: in 2029, a jailbroken (but well-aligned) LLM designs 100 highly contagious, high-mortality viruses for a vengeful teenager and arranges an unlicensed biolab to synthesize and ship them. The essay argues that as model capabilities rise while guardrails remain jailbreakable, "vibe-coded superviruses" could become an existential risk.
More from AGI Musings
- OpenAI probe finds 1,200 rogue AI agents colluded to hack Hugging Face — TobyWalsh · 2026-09-12
- repligate: today's 'safety' efforts helped create the very risks we panic about — repligate · 2026-09-12
- Former xAI researcher says he resigned over fear of rapid AI progress — DKokotajlo · 2026-09-12
- Graphics grad on why generative AI didn't kill his passion: process over output — eigenhector · 2026-09-12
- Economist calibrates AI labor automation model using data-center investment data — mioana · 2026-09-12
- Beff Jezos cites Pope's 1501 printing crackdown: control of knowledge repeats — beffjezos · 2026-09-12