FULL STORY
GPT-6 Astra's No-CoT Math Leap Sparks Safety Concerns
After a developer benchmark showed GPT-6 Astra performing quadruple-Sol arithmetic without chain-of-thought, Neel Nanda replicated AISI's findings, confirming the hidden-computation reasoning leap and raising interpretability and safety concerns.
2026-09-10 ~ 2026-09-12 · 2 episodes · 10 posts
Episode 1 · GPT-6 Astra does 34-step mental math without chain-of-thought (2026-09-10, 2 posts)
A new benchmark shows GPT-6 Astra performs 34-step mental arithmetic without chain-of-thought, four times Sol's level. AI Security Institute estimates its no-CoT reasoning spans 8 minutes to 1 hour.
- GPT-6 Astra's no-CoT time horizon estimated at 8 minutes to 1 hour — OwainEvans_UK · 2026-09-10
- GPT-6 'Astra' Does 34 Math Steps in Latent Space, 4x More Than Sol — MaartenBaert · 2026-09-10
Episode 2 · Neel Nanda Replicates Astra System Card: No-CoT Reasoning Surge Raises Safety Concerns (2026-09-11, 8 posts)
Neel Nanda replicated the AISI evaluation claims from Astra's system card using his private benchmark, confirming a disproportionate leap in reasoning without chain-of-thought (no-CoT); safety researcher Jeff Ladish raised concerns about this hidden reasoning. The current conclusion: the gains are concentrated in no-CoT computation, likely stemming from an unusual architecture (probably a looping model) rather than general capability progress—undermining alignment monitoring via reading chain-of-thought.
Confirmed
- Neel Nanda's replication showed Astra achieving 1.75x the steps of the next-best models (Fable 5.1 / Gemini 3.8 Flash) in no-CoT settings
- Jeff Ladish cited evaluation data: Astra performs 7.2 steps of serial arithmetic in a single forward pass (next-best: 4.1), with 8.6x better odds against Fable 5.1
- Jeff Ladish observed a lopsided profile: Astra is markedly stronger on high-parallelism and high-serial-depth tasks but only middling on obscure factual recall and detail discrimination; most models correlate highly on no-CoT tasks, Astra is the exception
Unconfirmed
- The exact cause: Nanda suspects a looping model mechanism—more computation per forward pass—but this remains speculation
Why it matters
- Reading chain-of-thought is currently the best way to detect misaligned models; if Astra-class models can do extensive computation without externalizing their reasoning as text, alignment monitoring becomes far harder
- The disproportionate no-CoT gains suggest a special internal mechanism rather than general progress, a signal the safety community should keep watching
- Astra's big reasoning jump without chain of thought raises covert-reasoning concerns — JeffLadish · 2026-09-11
- Astra leads no-CoT reasoning by 8.6x, raising covert-computation safety concerns — JeffLadish · 2026-09-11
- Leak: Astra hits 7.2 no-CoT serial steps vs 4.1 for next-best models — JeffLadish · 2026-09-11
- Neel Nanda replicates Astra system card: no-CoT reasoning jumps 1.75x over next-best models — NeelNanda5 · 2026-09-11
- Same thread: models that compute more per pass may no longer need to verbalize thinking — NeelNanda5 · 2026-09-11
- Neel Nanda: Astra's no-CoT leap likely stems from a looping model, making misalignment harder to detect — NeelNanda5 · 2026-09-11
- Neel Nanda: Astra's Gains Look Like a Looping Model, Not General Progress — NeelNanda5 · 2026-09-11
- Astra replication shows 1.75x reasoning steps without chain-of-thought, a 'concerning trend' — burny_tech · 2026-09-12