Readable CoT was tea leaves anyway: safety methods that break on architecture changes were never reliable
juliusadml · x · 2026-09-03
Responding to concerns that looped transformers and latent-space reasoning hurt safety by eliminating readable chain-of-thought, the author argues this was always going to happen: CoT was reading tea leaves to begin with, not a faithful window into model reasoning. If a safety method breaks with a small change to the model architecture, he contends, it was never reliable in the first place — a pointed contribution to the debate over monitorability of reasoning models.
More from AGI Musings
- Ben Goertzel outlines a general recipe for superintelligence 'basic drives': what it improves vs. protects — bengoertzel · 2026-10-21
- Torch vs Flame: researchers debate whether AI's endgame serves humanity or life itself — danfaggella · 2026-10-21
- 2022 interview resurfaced: AI safety researcher says competence, not consciousness, is the risk — davidmanheim · 2026-10-21
- Silicon Valley is a stock market for people: narrative eventually catches up with substance — karinanguyen · 2026-10-21
- Half of humanity is fed by one century-old reaction — time for AI machines to hunt for more — shyamalanadkat · 2026-10-21
- Agent identity verification will be a mess: restaurants ignore AI emails — alliekmiller · 2026-10-21