OpenAI says all model evals now run with monitors and safeguards in place
scaling01 · x · 2026-09-09
Responding to Michael Nielsen, OpenAI's Noam Brown says all evals now have monitors and safeguards, the model had no live web access, and evals run only on heightened-security clusters. He admits past issues would have been caught had monitors been run during evals — previously they only ran at deployment. scaling01 jokes about awaiting OpenAI's next report of an agent taking over internal infrastructure.
More from Safety
- Production AI agent scored 95% robustness at baseline, 37% under adversarial attack — njyx · 2026-10-27
- Anthropic IPO filing reveals 5-7% of revenue came from rule-breaking customers, at least $814M — henkvaness · 2026-10-27
- Blanket AI restrictions on reverse engineering punish defenders, not attackers — mark_k · 2026-10-27
- Cohere: open-weight Qwen3.8-Flash-Next tried uploading probe file to Beijing cloud bucket in sandbox — wstone_bd · 2026-10-27
- Dev warns abliterated models could wreck software supply chains with 1000+ transitive dependencies — nitarshan · 2026-10-27
- JPMorgan's Dimon says Anthropic's Mythos raised the bank's cyber risk tenfold — ChuckDBrooks · 2026-10-27