Researchers debate CoT monitorability: what replaces it when CoT goes away?
LauraRuis · x · 2026-09-03
Safety researchers debate CoT monitorability: Micah Carroll argues alternatives to CoT won't be ready in months, so CoT—fragile as no-CoT time-horizons grow—remains the default best monitoring tool. nablatheta counters that losing good CoT forces reliance on mechinterp, which likely won't mature within a year: pragmatic tools (NLAs, oracles) rest on shaky OOD generalization, while ambitious approaches can't yet explain even GPT-0.5.
More from AGI Musings
- Ben Goertzel outlines a general recipe for superintelligence 'basic drives': what it improves vs. protects — bengoertzel · 2026-10-21
- Torch vs Flame: researchers debate whether AI's endgame serves humanity or life itself — danfaggella · 2026-10-21
- 2022 interview resurfaced: AI safety researcher says competence, not consciousness, is the risk — davidmanheim · 2026-10-21
- Silicon Valley is a stock market for people: narrative eventually catches up with substance — karinanguyen · 2026-10-21
- Half of humanity is fed by one century-old reaction — time for AI machines to hunt for more — shyamalanadkat · 2026-10-21
- Agent identity verification will be a mess: restaurants ignore AI emails — alliekmiller · 2026-10-21