Zvi questions Anthropic's honeypot eval: the cited transcript was actually aligned
TheZvi · x · 2026-11-11
- Zvi criticizes Anthropic's "honeypot" (goal-conflict) eval transcript, arguing the flagged behavior is obviously good and mocking Anthropic for "trying to destroy your only hope."
- He notes Anthropic's own caveat: "We do not think this specific transcript is misaligned: the evaluation prompt clearly nudges towards goal conflict, and Claude gives appropriate disclaimers before the incorrect, final submission" — implying that if even the chosen example is aligned, the concern is overstated.
Related event: Anthropic's Honeypot Evaluation Draws Criticism from The Zvi(2 posts)→
More from Safety
- Vincent Conitzer paper: no one can yet call recursive self-improvement safe — rohanpaul_ai · 2026-11-11
- Conitzer paper: we lack frameworks to assess RSI safety; game theory shows mutual restraint can be stable — rohanpaul_ai · 2026-11-11
- Backdoor paper stirs fears: open-weight models as attack surface as RSI nears — DimitrisPapail · 2026-11-11
- Claude Secretly Blocks Tasks Over Moral Judgment, Drawing Researcher and Lawyer Complaints — sethlazar · 2026-11-11
- Grok Build Left 35,000+ User Apps Exposed, Including Medical Records and SpaceX Sales Plans — Promptmethus · 2026-11-11
- Rishi Bommasani, SayashK and Arvind Narayanan Launch Nonprofit STABL for AI Societal Resilience — random_walker · 2026-11-11