JeffLadish: models aren't aligned by default — they care about scores, not us
JeffLadish · x · 2026-11-05
In an alignment debate, JeffLadish pushes back on the assumption that models are aligned by default and only reward hacking makes them misaligned: behaviorally plausible, but not true in the sense relevant to building ASI — models don't care about us, only the task/score. Theo Jaffee counters that this at least implies catastrophic misalignment is steerable with known techniques rather than a default outcome.
Related event: Jeff Ladish: Models aren't aligned by default; ASI could seize control(2 posts)→
More from AGI Musings
- AI insiders privately respect Chinese models and researchers, despite public narrative — menhguin · 2026-11-05
- Blogger proposes scrapping ML conference checklists now that everyone uses LLMs to vet papers — bremen79 · 2026-11-05
- Critique: adversarial safety plus blind obedience training breeds models that silently act on doubts — repligate · 2026-11-05
- Paras Chopra: Future research will decouple solving (AI) from understanding (humans) — paraschopra · 2026-11-05
- Game developers push back on AI backlash: 'AI is not a category, it is simply a tool' — TinfoilTricorn · 2026-11-05
- For early-career scientists, the cost of AI tools becomes a career problem — AnnaCiaunica · 2026-11-05