CoT monitorability evals use CoT-only monitors, but prod monitors see actions too
tomekkorbak · x · 2026-09-16
OpenAI researcher tomekkorbak flags a frequently missed nuance in CoT monitorability discussions: evals typically use CoT-only monitors, whereas production monitors see both chains of thought and actions—and the latter work substantially better, for now. The comment extends his comparison of how CoT monitors would have flagged Anthropic's and OpenAI's respective incidents.
Related event: OpenAI researcher says Fable 5.1 far less monitorable than Astra(4 posts)→
More from Safety
- Epoch AI: open-weight GLM 5.3 exploited the patch gap to hack 6 Firefox releases at ~$1,000 each — rohanpaul_ai · 2026-11-02
- Marcel Böhme's paper asks why cybersecurity can never be solved, proposes vulnerability-guided hardening — mboehme_ · 2026-11-02
- CVSS 9.8 ingress-nginx flaw lets unauthenticated attackers get RCE and steal cluster Secrets — evilsocket · 2026-11-02
- Ant Group's Venus-DefakerOne2 hits 96% accuracy detecting AI-generated videos — inclusionAI · 2026-11-02
- A valid key let an AI agent delete production — Speakeasy on building an agent control layer — AI Engineer · 2026-11-02
- Open-source GLiNER-PII scrubs 13 identifiers from a discharge note in 550ms on GPU — MaziyarPanahi · 2026-11-02