Open-model RL practitioner pushes back on roon: safety standardization won't threaten open source

willcb · x · 2026-09-13

willcb, self-described spokesperson for the open model RL industry, says today's "pacing" debate poses no real threat to open source.

Key points:

He reads roon's claim as "open models will likely cause major security incidents leading to reactionary bans" — he disagrees, but argues safety standardization (which mainly affects the 2-5 big labs) actually makes such incidents less likely.
Enterprises don't want misaligned models, and de-risking alignment failures matters as training compute scales; China also doesn't want misaligned models, though alignment is a lower priority there.
The open world is still catching up and open models aren't yet dangerous enough to warrant heavy alignment compute; big labs doing the hard research and agreeing on protocols makes his job easier.

More from AGI Musings