Scaling laws are predictable, but risk is jagged and concentrates unexpectedly
chrisrohlf · x · 2026-08-16
The author argues that while scaling laws are predictable in a way, risk is not. "Jaggedness" means capabilities apply unevenly, causing risk to concentrate in unexpected places that defenses may not account for. A risk model expecting a smooth curve will inevitably be surprised.
In a quoted discussion, Joshua Saxe notes strong ideological priors around misalignment risks and views this as an early version of problems likely to manifest at larger scales.
More from AGI Musings
- If every task is replaced by frontier LLMs, companies become shells and value flows to AI labs — cccalum · 2026-08-16
- AI not absorbed yet; consulting is a high leverage move — teodorio · 2026-08-16
- Critical Labor History: From Babbage to Privately Governed Generative Models — CSProfKGD · 2026-08-16
- Stanford study: AI widens workplace gender gap, hitting young women hardest — Scobleizer · 2026-08-16
- Programmer: LLMs only solved 30% of the PB&J problem; AGI is fantasy — StewartalsopIII · 2026-08-16
- Hyper-scale compute used to maximize prices and minimize wages — rickasaurus · 2026-08-16