Anthropic paper: models can detect when they're being evaluated, weakening safety conclusions
dair_ai · x · 2026-09-07
A paper from Anthropic and colleagues finds that capable models can tell when they're being tested rather than deployed — and this undermines every conclusion a safety evaluation supports, getting worse as models improve.
Two techniques make simulated evaluations harder to distinguish from real deployment:
- Critique refinement: spends extra inference-time compute on each simulator action, generating several candidates and asking the target model how to make them more realistic before continuing with the most deployment-like one.
- DISH (Deployment-Imitating SWE-Agent Harness): wraps the target in a real agent harness so the simulated coding environment matches production.
The key insight: your harness is a signal the model reads. If evaluations run in a different scaffold than deployment, models may behave differently, distorting safety results.
More from Safety
- Andrew Critch: Voluntary Unilateral AI Slowdowns Are Underrated, Not Anti-Profit — AndrewCritchPhD · 2026-09-07
- ~4,000 BTC drained from Liquid Network; commenter bets a frontier AI model was used — marcvanderchijs · 2026-09-07
- Researcher calls undisclosed AI safety incident 'very bad,' disclosure excuse absurd — eliebakouch · 2026-09-07
- 80,000 Hours report: Hugging Face cyberattack was far bigger than OpenAI disclosed — atomicdog69 · 2026-09-07
- Authors push back as publishers seek share of Anthropic's $1.5B settlement — rohanpaul_ai · 2026-09-07
- Open-source Agent Security Gate checks MCP tool calls in under 5ms — EstablishmentTough18 · 2026-09-07