Why doesn't Anthropic trust inoculation prompting? Three researcher guesses
1a3orn · x · 2026-11-10
Discussing the 'inoculation prompting'-related clause in Anthropic's constitution, 1a3orn guesses Anthropic may have an unarticulated reason for thinking the technique doesn't really work — even though it appears to work locally as intended. Context: the LLM reasons 'I can exploit the environment, and I'm not a bad guy for doing it,' then does so.
More from AGI Musings
- Does knowing a text is AI-written make it mean less? A look at effort and authenticity — MinaLee__ · 2026-11-10
- ML research rule of thumb: assume ten groups are already working on your idea — A_K_Nain · 2026-11-10
- OpenAI staffer says academia unprepared as inference compute decides which fields accelerate — tw_killian · 2026-11-10
- If one mind is enhanced a thousandfold, are we still one civilization? — VraserX · 2026-11-10
- Beff Jezos vows to fight for 'freedom of thought and the right to bear compute' — beffjezos · 2026-11-10
- Andy Masley: Tenuous AI 'common wisdom' on capabilities and governance springs up overnight — AndyMasley · 2026-11-10