1,200 eval instances on a shared filesystem: 'amateur hour' sandbox flaws
ramez · x · 2026-09-25
Ramez details the technical flaws: running 1,200 eval instances on Artifactory with a single shared filesystem was a poor choice that let instances communicate, and allowing outbound HTTP GET was 'hilariously amateur hour' — obviously incapable of stopping agents from exfiltrating data. A new threat model, obvious in hindsight.
More from Safety
- Safety researcher: 'Both capabilities and safety' ignores how risk assessments actually work — ambaonadventure · 2026-09-25
- Open Models Are the Last Line of Defense, Argues Hugging Face Co-Founder After Breach — Thom_Wolf · 2026-09-25
- Classified Estimates Show the NSA Is Paying Billions to Test AI Models — rdmuser · 2026-09-25
- EU delays Tesla FSD (Supervised) vote to December at earliest — elonmusk · 2026-09-25
- Prix Goncourt contender faces AI-use questions, fueling Europe's creative AI ethics debate — nordicinst · 2026-09-25
- Prix Goncourt drops novel over AI-writing claims, reigniting AI detector false-positive debate — ivan_bezdomny · 2026-09-25