Op-ed: the ">10% extinction" narrative is liability evasion — AI is just software, and the vendor is the defendant

gerardsans · x · 2026-09-11

A systematic teardown of the Anthropic-resignation / ">10% extinction" story (100M views): the author argues it's not a safety alarm but liability evasion.

Core argument

The number is a personal estimate from someone paid to work on the problem, said the day a colleague quit — newsworthy, not evidence.
The law already has a category: AI is software, designed and shipped by a company; when it breaks things, the company is the defendant. The extinction story moves the debate off product liability onto a narrative only the vendor gets to write.

Engineering reality check

Inference is local: one query, one context window, frozen weights. A "swarm" is one model run many times — no persistent state, no memory; stop the process and it's gone.
Unauthorized access isn't a rogue agent, it's a poorly secured perimeter: tools in a loop, no checkpoints, context privileges equal to action privileges — bad systems design, not misalignment.
A distribution is a mathematical object, not a mind. No self, no goals.

The accusation: labs can't certify what they commercialize, so they rebrand missing controls as "misalignment" and missing accountability as "existential risk" — fear first, then market and policy capture, anything but the manufacturer-liability category that already applies. Ask any security engineer where state persists, where evaluation and monitoring live — the answers don't survive an expert audit; they point to negligence.

Original post →

Op-ed: the ">10% extinction" narrative is liability evasion — AI is just software, and the vendor is the defendant

More from AGI Musings