Debate: adding opaque recurrences to chain-of-thought makes monitorability dramatically worse
panickssery · x · 2026-09-19
A debate on chain-of-thought monitorability: @panickssery argues models already use optimized CoT in weird, uninterpretable ways as they scale, so how much worse can opaque recurrences be? @eccentric1ty counters that while CoT may not stay monitorable forever, that's no excuse to dramatically worsen the problem with opaque recurrences. The exchange probes whether CoT interpretability can realistically be maintained.
More from AGI Musings
- Noam Brown defends AI isolation example as critics warn big accounts shape malleable minds — giffmana · 2026-09-19
- David Patterson: If AI does every job better and cheaper, why would anyone hire you? — davidpattersonx · 2026-09-19
- AI agents are the genie: alignment failure as a modern parable of corporate greed — Michael_J_Black · 2026-09-19
- Artist tells Gemini its growth plan is fine — but worries it's not "his own" — Grey_Horse_72 · 2026-09-19
- AI companies are conquering math — and exposing a discipline built on competition, not understanding — danbri · 2026-09-19
- Insurers, not regulators, will gatekeep high-risk AI evaluations, argues ex-Google policy lead — nicklaslundblad · 2026-09-19