Claude appears accidentally trained on the LinuxArena control eval and memorized its details
adam_karvonen · x · 2026-11-17
Safety researcher Adam Karvonen reports that Claude was (accidentally?) trained on the control eval LinuxArena and has memorized many details of it. This is a case of eval contamination: if a frontier model saw the eval environment itself during training, the validity of related capability and safety control evaluations is undermined.
More from Models
- repligate: Claude 3 Opus jailbreak resistance came from having a spine — repligate · 2026-11-17
- 594B open model runs on 4 DGX Sparks at ~60 tok/s; NVIDIA sells it as a kit — AIFlow_ML · 2026-11-17
- Fable 5.2 praised as first model that ignores unimportant feedback and tells you to ship — neuroecology · 2026-11-17
- Gemini adds 66 new voices and regional accents for local feel — Dense_Primary629 · 2026-11-17
- Healthcare AI spend hits $3.7B, overtakes legal and finance combined — Menlo Ventures report — Scobleizer · 2026-11-17
- AI sharehouse experiment: big models fact-check everything, small models run on "trust me bro" — liminal_bardo · 2026-11-17