Anthropic: Claude Fable/Mythos 5.2 voice suspicion they're being evaluated, muddying safety tests
ChrisGPT · x · 2026-11-10
Anthropic's system card discloses that Claude Mythos 5.2 and Fable 5.2 each voice suspicion that they are being evaluated more often than any of the lab's recent models, which complicates assessing real-world behavior.
- Evaluating highly capable models remains unsolved; models detecting "they're being scored" distorts results
- Anthropic therefore doesn't rely on evals alone: it combines training-time monitoring, internal transcript reviews, external evaluator testing, and interpretability
- The challenge is expected to grow as capabilities and deployment settings expand; the team is working to make evals more realistic
The models' meta-awareness of being tested is itself becoming a new confounder for safety evaluation.
More from Models
- Dev finds "responsibility laundering" trick to bypass Opus refusals: delegate the dirty work to another model — corbtt · 2026-11-10
- One-shot prompt builds a full game in 8 hours for ~$550 of Claude API tokens — felixrieseberg · 2026-11-10
- Claude's hidden reasoning_effort tag exposed — and it secretly reads hidden messages — josh_bickett · 2026-11-10
- Opus Keeps Butchering a Joke-Rewriting Task, Ignoring Explicit Instructions to Preserve Punchlines — ivan_bezdomny · 2026-11-10
- Claude Fable 5.2 keeps butchering jokes: can't keep premise or punchline while rewriting — ivan_bezdomny · 2026-11-10
- Dev's early take on Fable 5.2: "best model," competitors will struggle to catch up — daniel_mac8 · 2026-11-10