Anthropic claims it works to keep eval environments unidentifiable to models
MaxKannen · x · 2026-09-11
Replying to concerns about models recognizing their benchmark environments, MaxKannen notes the environment may only be used for training rather than eval/benchmarking, and that Anthropic claims to put effort into making eval environments hard to identify.
Related event: Debate Over Eval Environment Contamination as Anthropic Defenses Questioned(3 posts)→
More from Models
- Dev review: Astra 'feels like Sol 5.7 on a good day', unusable for security work — jarrodwatts · 2026-09-11
- GPT-6 Astra docs draw attention: async tool calling and mid-turn steering point to multi-agent use — MikkoH · 2026-09-11
- Leaked GPT-6 Astra Scores 46% vs 12% for MolmoAct2 on Bimanual Robot Tasks — damianplayer · 2026-09-11
- OpenAI ships GPT-Live prompting guide: copying your old prompts won't hit SOTA — craigsdennis · 2026-09-11
- GPT-6 Astra burns through user's weekly usage cap, forcing a wait until Monday — max_paperclips · 2026-09-11
- Anthropic blocks minors from using Claude, HN debates age policy — petrusenko_max · 2026-09-11