CoT monitoring may fail: misaligned AI gets harder to detect, outpacing AI 2027
AaronBergman18 · x · 2026-09-02
Researcher thlarsen voices alarm at new developments: he previously thought that in short-timeline worlds, AIs would be misaligned but we'd get ample evidence from reading chains-of-thought (CoT), raising the odds of a reasonable response from labs and governments.
Now he still expects misalignment, but believes we'll lose the ability to tell via CoT — forced to rely on toolcalls and agentic behaviour instead — and even then investigation will be nearly impossible, since we'd have to trust the AI to self-report its reasoning. He adds this is another case of things moving faster than the AI 2027 scenario: Neuralese-style work started as early as March.
More from AGI Musings
- Essay: AI made test-style intelligence the cheapest commodity — judgment is what's left — r0ck3t23 · 2026-10-18
- "Yes, It's Lazy. That's the Point": Pushing Back on Humanoid Robot Housework Critics — VraserX · 2026-10-18
- Jason Furman: optimistic on AI productivity, skeptical firms can profit from it — sebkrier · 2026-10-18
- 562 AI PM job postings split into 4 roles — Model PM tops pay at $345K median — aakashgupta · 2026-10-18
- Longevity influencer: this is the last decade dying of old age is 'normal' — rand_longevity · 2026-10-18
- Only 85% of Anthropic hours and 76% of OpenAI workdays are agentic — yacineMTB · 2026-10-18