Ex-OpenAI researcher: kill-switch hopes died with GPT-3; open-source is safety's last hope
suchenzang · x · 2026-09-18
Former OpenAI researcher Suchenzang revisits how the AI race started: the metaphorical "kill-switch" died with the 2020 GPT-3 paper, as even well-intentioned safetyists (Anthropic implied) accelerated the race in their own quest for resources. ChatGPT's release killed it a third time — replicating GPT-N is now a national-interest topic in multiple countries. She mocks the shift from "kill-switches" to "pacing misalignment" talk, questions why extinction-level-IP companies stay in California where non-competes are unenforceable, and concludes open-source is the only remaining hope for safety-minded people to build real technical solutions on real models.
Related event: If AI IP Is Existential, Why Do Labs Stay in Non-Compete-Free California?(4 posts)→
More from AGI Musings
- Alpha School investigation: 17 campuses on probation, Bill Ackman as shadow lead, Holmes involved — benjaminjriley · 2026-11-02
- Google's chief economist: AI will transform more jobs than it kills — gains come from redesigning workflows — danielrock · 2026-11-02
- Daniel Lemire: AI is exploding demand for boring compute, server CPUs to double — lemire · 2026-11-02
- Alignment researcher: the word "aligned" implies consistencies that don't exist — davidmanheim · 2026-11-02
- By 2035, Indian Parents Will Ask How Many Agents You Manage, Not Your Salary — theteknosaur · 2026-11-02
- Under 5% of US consumers pay for AI, yet compute demand is already forcing usage cuts — NinaDSchick · 2026-11-02