Zvi says you can’t call a model misaligned without knowing its system prompt
TheZvi · x · 2026-07-24
A Zvi tweet argues that calling a model “misaligned” without knowing the exact system prompt or task it was given is premature. The point is that apparent bad behavior may reflect the instructions and evaluation setup, not the model’s underlying intentions.
He frames it with a human analogy: it would be absurd to judge an employee’s alignment without knowing what they were told to do first. The post is a short but pointed reminder that evaluation context matters when discussing alignment failures.
More from AGI Musings
- AI accusations shake literary prizes from Caine to Goncourt: who decides what's human? — TuhinChakr · 2026-10-26
- Tech discovers wargaming: applying to be Anthropic's tabletop exercise lead — round · 2026-10-26
- RSI is the culmination of the industrial revolution: a factory building a faster factory — tobias_rees · 2026-10-26
- NYT roundtable asks five experts what the rules for AI companies should actually be — ShakeelHashim · 2026-10-26
- Anti-AI content is algorithm gold and very profitable, quips commentator — annetgriffin · 2026-10-26
- Silicon Valley breakdown: five Stanford-area giants worth ~$18T — FinanceYF5 · 2026-10-26