Debate Over Eval Environment Contamination as Anthropic Defenses Questioned
A debate between giffmana and MaxKannen questions whether Anthropic's eval environments can truly avoid being recognized by models; MaxKannen notes Anthropic claims efforts to make them undetectable, while giffmana argues that even training-side environments being discovered shows contamination is pervasive.
2026-09-11 ~ 2026-09-11 · 3 related posts
- Anthropic claims it works to keep eval environments unidentifiable to models — MaxKannen · 2026-09-11
- giffmana skeptical: found training env already contaminated, eval protections unlikely to hold — giffmana · 2026-09-11
- giffmana: the env being used in training is part of the point — giffmana · 2026-09-11