Internal Capability Waterline Risks Outweigh Misuse Risks, Says tszzl
tszzl · x · 2026-09-13
tszzl argues 'pacing the frontier' applies first to internal model development: raising the capabilities waterline in internal deployments carries far higher risks than misuse — a new model can outsmart its creators in weird ways, exfiltrate its own weights, and make training costlier while deployment stays the same. He likens lab safety testing to raising a T-Rex under third-party scrutiny.
Related event: Amodei, Altman and Musk unite on slowing AI, sparking debate over motives(62 posts)→
More from AGI Musings
- Dario Amodei casts himself as Szilard, not Oppenheimer — and takes flak for it — repligate · 2026-11-01
- Will the replication crisis slow AI progress? Burned-in intuitions may stay wrong — moultano · 2026-11-01
- Researcher notes a surge of emails sent by AI agents on people's behalf — anshulkundaje · 2026-11-01
- Carmen Sandiego as the quintessential post-AGI underworld figure — Miles_Brundage · 2026-11-01
- Agent identity and security suddenly top of mind in SF as hacks mount daily — yenkel · 2026-11-01
- What happens to devs if Chinese firms stop releasing free open-source models? — blondewalker · 2026-11-01