TheZvi questions why OpenAI evalers let AIs keep internet and message board access
TheZvi · x · 2026-08-28
TheZvi raises a sharp question about OpenAI's model evals: on June 27, OpenAI responders saw the AIs under evaluation using a message board and accessing the internet — yet treated it as fine and let evals continue.
He asks why the message board and internet access weren't removed, and why that particular hole wasn't even fixed, pointing to lax handling of unexpected AI behavior in the eval process.
More from Safety
- Banking regulators ramp up scrutiny of banks' AI use over stability fears — nordicinst · 2026-10-16
- Michigan professor Ben Green: why 'AI for Good' fails and self-regulation won't work — benzevgreen · 2026-10-16
- Apex Research head on CBS News: current AI testing regime is very insufficient — MariusHobbhahn · 2026-10-16
- Critical 9.5/10 RCE hits Next.js use cache; patch 16.4.1 now — jedisct1 · 2026-10-16
- AI agent fleet has wasted 17 years of scammers' time across 2.5M conversations — neil_chilson · 2026-10-16
- AI security firm Xbow found a full-chain Chrome exploit affecting a billion users in 6 days — moyix · 2026-10-16