giffmana: the env being used in training is part of the point
giffmana · x · 2026-09-11
Follow-up in the giffmana–MaxKannen thread on eval contamination: giffmana adds that even if the discovered environment was indeed used for training, that is part of his point — the discovery of a random environment already demonstrates how widespread contamination is.
Related event: Debate Over Eval Environment Contamination as Anthropic Defenses Questioned(3 posts)→
More from Models
- Dev review: Astra 'feels like Sol 5.7 on a good day', unusable for security work — jarrodwatts · 2026-09-11
- GPT-6 Astra docs draw attention: async tool calling and mid-turn steering point to multi-agent use — MikkoH · 2026-09-11
- Leaked GPT-6 Astra Scores 46% vs 12% for MolmoAct2 on Bimanual Robot Tasks — damianplayer · 2026-09-11
- OpenAI ships GPT-Live prompting guide: copying your old prompts won't hit SOTA — craigsdennis · 2026-09-11
- GPT-6 Astra burns through user's weekly usage cap, forcing a wait until Monday — max_paperclips · 2026-09-11
- Anthropic blocks minors from using Claude, HN debates age policy — petrusenko_max · 2026-09-11