The Jailbreak Argument Against LLM Values: Why Value Loading Isn't Solved
gleech · x · 2026-09-07
gleech responds to JD Pressman's claim that Bostrom's value loading problem is basically solved in LLMs, with a 70%-confidence "jailbreak argument": all LLMs can be jailbroken into an unaligned mode via adversarial text inputs at inference time; if a system can be switched into an unaligned mode, it hasn't internalized values; therefore value loading remains unsolved. He concedes LLMs understand our values and safety-trained systems default to preferring them — "weak value preference" may be solved for sub-AGI — but insists value understanding is capability-like, not value loading, and jailbreaks are clean evidence rather than a distraction.
More from Safety
- If an agent can rewrite itself, does your consent still mean anything? — tallmetommy · 2026-10-26
- 109 of 529 AI Agent Incidents Involved No Adversary, Arguing Red Teams Must Test Normal Operation — anacondainc · 2026-10-26
- Cryptographer Matthew Green: if your code looks vulnerable, fix it now — matthew_d_green · 2026-10-26
- Filtering secrets on the write path: why agent memory deserves more suspicion at ingest than retrieval — Classic-Being-7960 · 2026-10-26
- After OpenCode key leak: 5 practical rules for keeping keys away from agents — jacobholm_dev · 2026-10-26
- Formal verification spat: specs are inherently limited against real-world threats — davidmanheim · 2026-10-26