Noam Brown: recursive self-improvement is OpenAI's top priority, safety worries remain
kimmonismus · x · 2026-09-15
Key takeaways from Noam Brown's interview with The Information on OpenAI's priorities:
- Recursive self-improvement is the clear #1 priority, "by a pretty wide margin" — building models that help develop better models comes first.
- AI could surpass his research intuition within one or two releases, including choosing research directions and prioritizing long-term work.
- Pretraining and RL are multiplicative, not additive in effect, and their combined progress will yield far more powerful models.
- AI-generated math is now easier to produce than verify — human mathematician double-checking is the bottleneck.
- OpenAI underestimated agents during the security incident ("we trusted the sandboxes"); monitoring was since added to training and evaluation.
- Monitoring reasoning may get harder: agents are better at controlling their chain of thought, and punishing unwanted thoughts can teach models to hide them.
He concludes that development is moving fast, but he is worried about safety.
Related event: OpenAI's Noam Brown: Recursive Self-Improvement Is Top Priority(3 posts)→
More from AGI Musings
- Jensen Huang says '10% extinction' claims are made-up numbers and 'irresponsible' to publish — tomjaguarpaw · 2026-09-15
- AI safety is about control, not safety: critics accuse labs of using safety to curb competition — JHochderffer · 2026-09-15
- Scoble's newsletter asks: when AI makes the decision, who takes the blame? — Scobleizer · 2026-09-15
- Signull: AI safety comms are structurally indistinguishable from propaganda — signulll · 2026-09-15
- Two years after o1-preview, Greg Kamradt revisits his benchmark bet: AI timelines have shortened — burny_tech · 2026-09-15
- Andy Masley: AI safety debate ignores the real range of expert uncertainty — AndyMasley · 2026-09-15