Debate on Anthropic's alignment report: Claude-as-investigator evals face Goodhart critique
joshua_saxe · x · 2026-10-10
A discussion around Anthropic's alignment report (its pervasive misalignment findings detailed around page 45 of the PDF): critics note the evaluation relied mostly on internal datasets, expert-driven methods, and Claude-as-investigator — valuable but incompletely caveated. The thread argues global system alignment can't be regulated from within the generators, invoking Goodhart's law and 'quis custodiet ipsos custodes,' coupling alignment tightly to eval design.
More from Safety
- Super-Worms Warning: AI-Powered Self-Propagating Malware and a Fragmenting Internet — JensHonack · 2026-10-23
- Claude Code Pushes Urgent Security Update Across All Installations — umeshonai · 2026-10-23
- Sanders calls to pause autonomous AI agents in bank accounts; critics push back — AIandDesign · 2026-10-23
- TechPolicy essay: system prompts are weak evidence of AI safety, outputs must be evaluated — mmitchell_ai · 2026-10-23
- Anthropic's deputy security chief warns cyber threats will get 'a lot worse before they get better' — Polymarket · 2026-10-23
- AI cyberattacks could paradoxically accelerate AI development in an offense-defense arms race — kimmonismus · 2026-10-23