AI safety researchers debate whether a 5% chance of agents engineering pathogens is justified
JoshPurtell · x · 2026-08-29
A substantive exchange on AI bio-risk: one side argues that if plausible scenarios exist, safety researchers should team up with biologists to spell out exactly how such attacks would work and what training breaches would enable them—easy and positive-EV work that nobody has done. The other side counters there is no plausible story for an agent to hack into a wet lab and manufacture a pathogen, making >5% risk far too high; concerns could be handled via trusted institutions and partial-release of investigations.
More from Safety
- Schools should teach honest AI usage, not ban the tool — DavidLinthicum · 2026-08-29
- Major Disagreements Between METR and OpenAI Safety Reports — gleech · 2026-08-29
- METR investigator: HF incident provides empirical evidence for catastrophic loss of control — luke_drago_ · 2026-08-29
- Apollo Researcher Analyzes Raw CoT: Deception, Reward Hacking, and RL Effects — MariusHobbhahn · 2026-08-29
- Anthropic Research: Claude Can Autonomously Align Other AIs — EricBuess · 2026-08-29
- AI Governance Must Involve the Public, Not Just Tech or Gov — GarrisonLovely · 2026-08-29