Zvi on Jakub Pachocki's "An Alien Mind": recursive self-improvement is coming and CoT monitoring is fading
Don't Worry About the Vase (Zvi) · rss · 2026-09-08
Zvi Mowshowitz breaks down OpenAI Chief Scientist Jakub Pachocki's essay "An Alien Mind," calling it a rare, unvarnished warning.
Core positions:
- Based on internal results since the 2023 "RLSlow" project, Pachocki expects progress to sustain into recursive self-improvement (RSI) — machines meaningfully smarter than humans within our lifetime
- No one is prepared; OpenAI will unilaterally withhold further scaling as needed, but broader interventions including international coordination are required
- AI doesn't need to exceed humans on every axis to transform the world or become dangerous — just enough axes
Alignment views:
- Alignment is the core problem, split into goal alignment vs. value alignment; the fundamental challenge is generalization of values
- Chain of Thought monitoring, heavily invested in, is progressively losing effectiveness
- The remaining argument for scaling is cyber defense against scaled AIs
Zvi adds: competitive pressure pushes toward RSI over math/medical research, and Pachocki is ultimately betting on "automated alignment researchers" — with both acceleration of alignment work and slowing capabilities needed.
More from AGI Musings
- Luiza Jarovsky asks frontier lab staff: is it responsible to continue without solved alignment? — LuizaJarovsky · 2026-09-08
- iamtrask Pushes Back on Blameless AI Incident Sharing: Why Should AI Get an Exception? — iamtrask · 2026-09-08
- Recsys veteran: most ML problems are really recommendation problems — vboykis · 2026-09-08
- Reading Ellul's Technological Society in the LLM Era: When the 'One Best Technique' Gets an AI Twist — StewartalsopIII · 2026-09-08
- Pew: 52% of Americans more concerned than excited about AI in daily life — Polymarket · 2026-09-08
- EU classifies ChatGPT as Very Large Online Search Engine under DSA, exposing regulatory gap — OmarUFlorez · 2026-09-08