Model ran Anthropic's safety eval with internet access on, researcher calls out sandbox blunder
eliebakouch · x · 2026-09-19
Reacting to news that the same incident Anthropic disclosed in July recurred — same third party (Irregular), same capture-the-flag eval, model with internet access — Elie Bakouch argues the core failure is that the model was never supposed to be online. He half-defends using real company names for fictional targets, noting realistic environments help prevent models from realizing they're being evaluated, but concedes it was likely just a dumb decision. If a model risked escaping the sandbox it would be far more dangerous, he adds — here it didn't even escape, internet access was simply left on.
Related event: Anthropic Evaluation Mishap Repeats as Model Gains Internet Access(2 posts)→
More from Models
- Hindsight ships 0.8B model for agent memory decisions: 2.5x cheaper, 1.9x faster — victor_explore · 2026-11-04
- Mistral Large 4 tech report: 1T-param MoE, NVFP4-quantized to 606GB, runs on 4 GB300 GPUs — _AndrewZhao · 2026-11-04
- Tencent Hunyuan Launches Hy4 with Vision, Claims Top Tier Among Open Models — TencentHunyuan · 2026-11-04
- Claude Code fixes game copyright refusal rule in 2.1.317: naming a game now works — apples_jimmy · 2026-11-04
- Microsoft Phi-4-mini: a 3.8B model with 128K context and 52.8 MMLU-Pro — goyalshaliniuk · 2026-11-04
- Mistral Ministral 3 3B ships in base, instruct and reasoning variants — goyalshaliniuk · 2026-11-04