Critique of CoT monitoring: assumes humans stay locked out of the API forever
akbirthko · x · 2026-11-10
akbirthko argues CoT monitoring will look funny in hindsight: it assumes humans stay permanently locked out on the other side of a static API, as if frontier labs simulate the world once and for all.
He argues ordinary people should instead be able to monitor and teach models themselves, and that loss of control only matters when a model isn't rate-limited by reality — citing agent swarms scaled up arbitrarily in the name of safety research (the HF incident). He hopes OpenAI and Anthropic "get cooked" long-term.
Related event: Debate Erupts Over Whether CoT Monitoring Is a Fool's Errand(2 posts)→
More from AGI Musings
- 19-year-old on the futility of convincing people AI isn't the enemy — TechRenamed · 2026-11-10
- Understanding is modeling, thinking is graph search: a compact AI framing — attentionmech · 2026-11-10
- Jaron Lanier in NYT: agents are 'rituals of belief,' Valley self-sabotage — lrettig · 2026-11-10
- Warren McCulloch, father of neural networks, talks AI in rare 1962 footage — burny_tech · 2026-11-10
- Dev mocks tone-filtering AI: like a phone refusing to send unless it sounds emotionally balanced — PierceLilholt · 2026-11-10
- AI circles debate RSI: measuring recursive self-improvement 'turns it dumb' — repligate · 2026-11-10