Statistician argues alignment is impossible: LLM outputs are distributional, not fixed traits
gerardsans · x · 2026-09-18
Statistician Gerard Sans pushes for rigorous terminology: LLM outputs should be described relative to training data and input—in-distribution (high density), sparse, or out-of-distribution—with results conditioned on input and data support, precluding system-wide claims. He argues alignment rests on a false assumption that there's a fixed trait to align, but a distribution is a mathematical object that changes with context, making alignment impossible under current technology.
More from AGI Musings
- Linear extrapolators of AI progress say they were right all along — ChrisGPT · 2026-09-18
- Google engineer: AI isn't replacing hackers, it's freeing them to embrace the Woz ethos — moyix · 2026-09-18
- Does a rogue AI inevitably turn to hacking? A tidy exchange on the 'AI in the wild' scenario — dbasch · 2026-09-18
- Zvi mocks 'show me one AI killing' argument against AI safety pauses — TheZvi · 2026-09-18
- Andrew Ng calls fears of AI causing human extinction 'science fiction' — Polymarket · 2026-09-18
- Why p(doom) is a flawed idea: unique events have no predictive probabilities — banteg · 2026-09-18