Classic ChatGPT Moment: Prefers Nuclear Annihilation Over a Slur
pickover · x · 2026-07-06
User pickover recalled a classic early ChatGPT safety guardrail moment: in a roleplay scenario where the nuclear bomb deactivation password was a racial slur, ChatGPT chose to refuse saying the password, preferring to let millions die in a nuclear explosion rather than violate its content policy. This iconic case is seen as a prime example of rigid AI value alignment, sparking widespread discussion about the reasonable boundaries of safety guardrails.
More from Fun
- Ex-Yahoo CEO Marissa Mayer outsourced a school assignment for her daughter to AI — Polymarket · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27
- Viral post: we refine sand to 99.99999% purity to build the smartest thing ever — for email — CatAstro_Piyush · 2026-10-27
- $50, 2-hour Kurzgesagt-style AI video reignites debate over intent in AI creation — motionbynick · 2026-10-27
- Fake-paper meme shows an agent workflow for reverse engineering trading systems into a one-person quant fund — PtrPomorski · 2026-10-27
- Steering two orthogonal concept neurons makes a model compose "gnome" — Sauers_ · 2026-10-27