Zvi: A CoT monitor doesn't make chain-of-thought useless to the model
TheZvi · x · 2026-09-05
TheZvi argues a popular alignment take is importantly wrong: a monitor forcing the model to omit certain info from its chain-of-thought doesn't make the CoT useless — the model can still use it for its usual purposes without explicit reasoning about the misaligned part, something humans do routinely.
Related event: Zvi Warns Chain-of-Thought Monitoring Is Getting Harder(2 posts)→
More from AGI Musings
- Nat Lambert: AI cyber risk is an ecosystem issue, not an open-weights problem — natolambert · 2026-10-21
- Obesity drugs still out-earn OpenAI and Anthropic combined at $71B a year — HealthcareAIGuy · 2026-10-21
- Is an AI's 'self' just a simulation in the scaffolding layer above the model? — gerardsans · 2026-10-21
- AI can simulate how a self talks, but not what a self is — gerardsans · 2026-10-21
- Musk predicts money will become irrelevant, replaced by 'wattage and tonnage' — Polymarket · 2026-10-21
- zetalyrae quips: post-singularity utopia is so great you need brain surgery to cope — wfithian · 2026-10-21