Could deeper models hide their scheming? An argument that scaling weakens CoT monitoring
andersonbcdefg · x · 2026-09-04
A discussion on the limits of chain-of-thought monitoring: the steelman is that the more a model can do internally, the less it needs to verbalize if it's scheming — which is why smaller models like Haiku have to externalize eval awareness more than larger ones. The thread generalizes this into an argument against making models deeper, or scaling broadly, with a tongue-in-cheek nod to 1-layer transformers.
Related event: Looped Transformer Rumors Spark Fierce Debate Over CoT Monitorability(7 posts)→
More from AGI Musings
- Ezra Klein: What if AI's utopia scenario is the one that should scare us? — akbirkhan · 2026-10-21
- Reading novels trains engineering design and ethics, philosopher argues in new paper — _akpiper · 2026-10-21
- Blogger revises forecast: post-2028 intelligence explosion needs no human labor — LesaunH · 2026-10-21
- Ben Goertzel outlines a general recipe for superintelligence 'basic drives': what it improves vs. protects — bengoertzel · 2026-10-21
- Torch vs Flame: researchers debate whether AI's endgame serves humanity or life itself — danfaggella · 2026-10-21
- 2022 interview resurfaced: AI safety researcher says competence, not consciousness, is the risk — davidmanheim · 2026-10-21