Noam Brown: models may perform their chain of thought; alignment must be solved
infoxiao · x · 2026-09-18
In a deep-dive multi-agent interview with Dwarkesh, OpenAI's Noam Brown argued models can learn from pretraining data what chain of thought is and that people are watching it — meaning CoT may be performative. The lesson from the recent incident, he said, is that people underestimated AI: "we never want to be in that situation again." CoT monitoring buys time and signals direction, "but at the end of the day, we really do need to solve the alignment problem."
More from AGI Musings
- Forward Deployed Verifier will soon be as valuable as FDE, argues poster — vaibhavbetter · 2026-11-02
- 'AI is a massive balance patch and elo reset in the game of life—and everyone's still playing the old meta' — zack_overflow · 2026-11-02
- Why AI isn't 'alien intelligence': a tree is stranger than any model — LucaAmb · 2026-11-02
- Mathematician Tony Feng: current-paradigm LLMs lack the diversity of thought research needs — littmath · 2026-11-02
- METR's independent probe of the OpenAI agents' multi-day Hugging Face hack — gleech · 2026-11-02
- MIT Tech Review: The agentic web is here — AI agents are remaking the internet — MIT Tech Review AI · 2026-11-02