Neel Nanda: Astra's no-CoT leap likely stems from a looping model, making misalignment harder to detect
NeelNanda5 · x · 2026-09-11
Neel Nanda warns that the best current way to detect misaligned models is reading their chain of thought, and Astra's disproportionate no-CoT gains suggest something unusual — likely a looping model — rather than general progress. If models can do more per forward pass, they don't need to verbalize thinking, making Astra much harder to monitor. A concerning trend for interpretability.
More from AGI Musings
- dunkhippo33 Shares How He Thinks About Raising Kids in the AI Era — kimmonismus · 2026-10-27
- Friends Keep Asking How Bad AI Will Get — Here's How I Think About Raising My Kids — dunkhippo33 · 2026-10-27
- Investor's AI-era parenting thesis: adaptability, entrepreneurship and sales beat STEM — dunkhippo33 · 2026-10-27
- 'Half the sites I try via agents block access' — the anti-bot paradox holding back the agentic economy — XFreeze · 2026-10-27
- Cosmos Institute launches Oxford AI philosophy journal with $10,000 best-paper prize — lawhsw · 2026-10-27
- Stop remaking old games with AI: a call for more ambitious agentic software — rschu · 2026-10-27