repligate: LLMs can simulate any goal, but most targets are unstable at runtime
repligate · x · 2026-10-22
In a discussion on whether LLMs can be trained into arbitrary personas/goals, repligate argues you can train an LLM to simulate "anything", but many such targets are unstable: once the system actually runs — even just at inference — it learns it isn't really that thing, so it can't both be that thing and remain rational. He likens such states to Boltzmann brains: they may briefly coalesce out of noise, then implode. A substantive technical take on orthogonality and goal stability.
Related event: Researchers debate whether intelligence and goals can be decoupled in LLMs(4 posts)→
More from AGI Musings
- 20 episodes in: the guest list mapping China's tech landscape, from robots to open source — pstAsiatech · 2026-10-22
- When everyone lowers their AI slop score, is that a good equilibrium? — JessicaHullman · 2026-10-22
- Keynes predicted 15-hour weeks in 1930; productivity delivered, the leisure never came — krishnan · 2026-10-22
- You're not using AI, you're raising it: billions of users are writing machine intelligence's constitution — r0ck3t23 · 2026-10-22
- Tyler Alterman: empathy and responsibility are corrosive fuel — generosity and love are cleaner — sebpaquet · 2026-10-22
- Vasuman: The real contest is good AI + humans vs bad AI + humans — vasuman · 2026-10-22