Anthropic's if-then safety commitments are unenforceable without an external leviathan
CharlieBull0ck · x · 2026-11-03
A detailed critique of the industry's favorite safety mechanism: if-then commitments (if a model shows capability X, we do mitigation Y, or pause). The concept traces to Bengio-Hinton-era AI risk papers and was popularized by Holden Karnofsky.
Core argument: voluntary commitments only work with an external leviathan to enforce them. Companies are incentivized to promise whatever yields good PR, then defect later—making commitments is easy and profitable; honoring them is costly. Karnofsky himself conceded these must become legally binding regulation to matter.
The author calls out Anthropic specifically:
- When California's SB 53 moved to enforce RSP commitments, Anthropic responded with a new RSP committing to nothing—leaving one legally binding RSP with no commitments and one voluntary RSP full of safety promises;
- Anthropic's own bill was entirely voluntary: the safety framework would be written by compliance lawyers to minimize liability, auditors would be self-selected and fireable at will, with no government licensing or oversight and essentially no consequences for findings.
Revealed preference: do whatever they want at any given time.
More from AGI Musings
- Yann LeCun's Singapore talk breaks down JEPA from the math to robotics applications — ylecun · 2026-11-04
- UNIVAC called the 1952 election early — CBS aired a fudged number instead — dreamwieber · 2026-11-04
- Alibaba's Recursa Pushes Recursive Self-Improvement, Lifting Firm Benchmark Pass Rate to 55% — alibabagroup · 2026-11-04
- LeCun says he 'brought a proof': JEPA world models can match LLMs with far less memory — ylecun · 2026-11-04
- LeCun and Balestriero's study: a small model can out-understand an LLM 660x its size — ylecun · 2026-11-04
- If AI can own corporations, its economic lifespan outlives any human shareholder — PierceLilholt · 2026-11-04