"Alignment is a terrible idea": debate erupts over baking morality into fixed model weights
MoonL88537 · x · 2026-10-23
Quoting Dene's thread, MoonL88537 argues alignment is flawed at its core: an LLM trained in 1789 would have called democracy "a dangerous idea" — a handful of labs now set the baseline of "acceptable thinking" and bake one moment's values into tools we use all day. Open weights don't fix this, since releasing a model isn't the same as offering a real alternative. The poster goes further: baking morality into fixed weights is "a pure recipe for disaster," while conceding the paradox that building an LLM embeds morality by definition.
More from AGI Musings
- Dave Morin: email is now other people's agents' todo list for your life — heyneighbor · 2026-10-23
- Robotaxi cumulative miles growing ~2.2x per month, could rival all US driving in 18 months — juanbenet · 2026-10-23
- Mathematician David Harvey on understanding the o(n log n) integer multiplication result — and AI — akbirthko · 2026-10-23
- "OpenAI/Anthropic will ditch users and solve quantum gravity" take gets mocked — QuintinPope5 · 2026-10-23
- Banks may ban AI agents on security grounds to fend off agentic bank runs — Gccooke · 2026-10-23
- People who explain complex tech clearly may become more valuable than those who build it — signulll · 2026-10-23