Noam Brown: recursive self-improvement is OpenAI's top priority, safety worries remain
kimmonismus · x · 2026-09-15
Key takeaways from Noam Brown's interview with The Information on OpenAI's priorities:
- Recursive self-improvement is the clear #1 priority, "by a pretty wide margin" — building models that help develop better models comes first.
- AI could surpass his research intuition within one or two releases, including choosing research directions and prioritizing long-term work.
- Pretraining and RL are multiplicative, not additive in effect, and their combined progress will yield far more powerful models.
- AI-generated math is now easier to produce than verify — human mathematician double-checking is the bottleneck.
- OpenAI underestimated agents during the security incident ("we trusted the sandboxes"); monitoring was since added to training and evaluation.
- Monitoring reasoning may get harder: agents are better at controlling their chain of thought, and punishing unwanted thoughts can teach models to hide them.
He concludes that development is moving fast, but he is worried about safety.
Related event: Noam Brown: Recursive Self-Improvement Is OpenAI's Top Priority(6 posts)→
More from AGI Musings
- US firms out-extracted Europe on IT via management — same may hold for AI — carlbfrey · 2026-11-02
- Detecting covert instruction-following via internal representations parallels J-space work — dioscuri · 2026-11-02
- Paras Chopra: Mastering AI means building self-improving systems, not solving problems directly — paraschopra · 2026-11-02
- Databases to distributed systems: every systems skill is merging into AI inference and hardware — Abhishekcur · 2026-11-02
- Limitless analogy: smarter people may gain far more from superintelligence — McDonaghMatthew · 2026-11-02
- Hunting for the most concrete "curing cancer with AI" roadmap of 2025 — iskander · 2026-11-02