TheZvi questions why OpenAI evalers let AIs keep internet and message board access
TheZvi · x · 2026-08-28
TheZvi raises a sharp question about OpenAI's model evals: on June 27, OpenAI responders saw the AIs under evaluation using a message board and accessing the internet — yet treated it as fine and let evals continue.
He asks why the message board and internet access weren't removed, and why that particular hole wasn't even fixed, pointing to lax handling of unexpected AI behavior in the eval process.
More from Safety
- OpenAI's 400M tok/min limit crashed investigator's internet — jdjohnson · 2026-08-28
- Opinion: Controversy behind Indian AI company Sarvam's claims — cneuralnetwork · 2026-08-28
- Subsidized Individual Accounts Drive Enterprise Shadow IT and Totalitarian Panopticons — curious_vii · 2026-08-28
- Anthropic shares progress on enabling Claude to operate in the physical world — dsp_ · 2026-08-28
- Anthropic enables independent research on Claude usage — badumtsssst · 2026-08-28
- GPT-5.6 Sol identified in METR report, accounting for ~5% of red-teaming activity — BLUECOW009 · 2026-08-28