OpenAI's Astra reported to use 'opaque reasoning' architecture, harming safety oversight
thlarsen · x · 2026-09-02
Ryan Greenblatt comments on reports that OpenAI's new model, Astra, uses an 'opaque reasoning' architecture where reasoning occurs in activations rather than natural language. He labels this potentially the single worst development for AI safety to date.
Key Points:
- Oversight Risks: This architecture could significantly impair supervision and monitoring, as reasoning processes are no longer fully visible in natural language Chain-of-Thought (CoT).
- Current Status: The article suggests recurrent depth is limited, meaning the model still relies on natural language CoT, though to a reduced extent, making text-based monitoring much less useful.
- Future Concerns: A natural progression scaling up opaque reasoning could lead to models reasoning entirely in latent space, likely destroying the usefulness of natural language reasoning for oversight.
More from AGI Musings
- When AI can clone your product by sight, paid software's moat is service and trust — sujingshen · 2026-10-18
- Anthropic borrows lawyers' ethics for Claude agents that closed 186 deals with no human approval — rohanpaul_ai · 2026-10-18
- Tim Sweeney: AI will unlock a new golden age of games too costly to build before — TinfoilTricorn · 2026-10-18
- Researcher argues we may solve AI-human alignment before humans align with each other — 3scorciav · 2026-10-18
- EigenGender: private-industry coup isn't the default, but worth guarding against — EigenGender · 2026-10-18
- Writing instructor's meltdown: "LLMs are literally just spicy autocomplete" — Decent-Decision-9028 · 2026-10-18