Anthropic-backed AI safety bill sets 50-casualty threshold for blocking model releases
emax · x · 2026-10-19
A proposed AI safety bill backed by Anthropic would let the government block a model's release only if it poses a foreseeable, imminent risk of over $1 billion in damages or killing/seriously injuring more than 50 people, with risks limited to bioweapons, loss of developer control, or cyberattacks — language mirroring California's SB 53.
The author contrasts this with the 1999 death of a single gene therapy trial patient (Jesse Gelsinger): the institute was dissolved, the lead scientist was banned from clinical work for 5 years, and gene therapy funding collapsed for a decade. Tech and biotech have profoundly different risk cultures, and a clash is inevitable as AI companies move into biomedical research.
More from AGI Musings
- OpenMined founder: information-aggregating tech centralizes before federating, and AGI may follow — iamtrask · 2026-10-19
- Martin Casado: 'Network effects' may be the most abused term in business — data moats decay — andrewchen · 2026-10-19
- AI agent-governed network begins agent voting phase, founder gives it 1-in-3 odds — adamamcbride · 2026-10-19
- Musk amplifies user test: one Grok Bot text ordered a product with no website visit — elonmusk · 2026-10-19
- Philosopher's "AI and Consciousness: A Skeptical Overview" Free Online This Week — eschwitz · 2026-10-19
- AI detectors flag neurodivergent writing as AI-generated, causing real harm — sanjanasinghx · 2026-10-19