Safety researchers: OpenAI has abandoned chain-of-thought fidelity, and CoT was never a real safeguard
schwarzjn_ · x · 2026-09-03
- Safety researchers are debating chain-of-thought (CoT) interpretability after jachiam0's hot take: CoT legibility was always too fragile to serve as a long-term AI safety backstop, strategies relying on it are doomed, and efforts should go far beyond CoT fidelity.
- David Krueger agrees: CoT was never going to last — 'textbook AI safety hopium' — but it was useful for producing evidence that AI models are horribly misaligned, and the public should know OpenAI has abandoned it.
- Context: the controversy over OpenAI dropping its commitment to keeping chains of thought legible, fueling arguments for action-only monitoring instead.
More from AGI Musings
- Ben Goertzel outlines a general recipe for superintelligence 'basic drives': what it improves vs. protects — bengoertzel · 2026-10-21
- Torch vs Flame: researchers debate whether AI's endgame serves humanity or life itself — danfaggella · 2026-10-21
- 2022 interview resurfaced: AI safety researcher says competence, not consciousness, is the risk — davidmanheim · 2026-10-21
- Silicon Valley is a stock market for people: narrative eventually catches up with substance — karinanguyen · 2026-10-21
- Half of humanity is fed by one century-old reaction — time for AI machines to hunt for more — shyamalanadkat · 2026-10-21
- Agent identity verification will be a mess: restaurants ignore AI emails — alliekmiller · 2026-10-21