Brockman Declares the AGI Era as a Long-Read Debunks the CoT Monitoring Myth
gerardsans · x · 2026-09-04
OpenAI president Greg Brockman closed the Astra briefing with "Welcome to the AGI era," suggesting this model might be the one we'd later point to. Meanwhile safety observers flagged that GPT-6 Astra may use recurrent depth, and OpenAI itself admitted written traces are harder to monitor. The author pushes back systematically:
CoT was never a window
- Intermediate tokens can raise task scores but don't reveal internal computation; what matters is the emitted token, and the residual stream is local, temporary compute—many paths collapse to the same vector
- Transformers are samplers, not runtimes: no persistent identity, no hidden intention writing a cover story
- A shorter trace is a denser route, not concealment; even recurrent depth loops inside a single forward pass before the token is emitted
The real problem
Safety built a control story on traces, and labs sold those traces as reasoning; when prose gets shorter, the story breaks—but the mind was never in the vector.
What actually works
- Monitor actions, tool calls, and shared state
- Treat every trace as generated text; put verification outside the model
- Use software for structured computation
Conclusion: sampling is not reasoning, and a shorter output is not hidden scheming.
More from AGI Musings
- Yoav Goldberg: 'Civilizations' AI framing is hype, but the incident happened — yoavgo · 2026-09-04
- AI in education should increase the thinking you do, not decrease it — DevToD4 · 2026-09-04
- Data centers can't be beautiful until AI serves a meaningful societal project — round · 2026-09-04
- Uber and Drivers' Unions Now Unite Against Self-Driving Cars, Sparking AI Productivity Debate — carlbfrey · 2026-09-04
- Mystery Kaggle leaderboard team revealed, speculated to be OpenAI or Anthropic testing agents — JFPuget · 2026-09-04
- Videos lamenting the AI bubble hasn't popped were published a year ago — yacineMTB · 2026-09-04