Neel Nanda Replicates Astra System Card: No-CoT Reasoning Surge Raises Safety Concerns
Neel Nanda replicated the AISI evaluation claims from Astra's system card using his private benchmark, confirming a disproportionate leap in reasoning without chain-of-thought (no-CoT); safety researcher Jeff Ladish raised concerns about this hidden reasoning. The current conclusion: the gains are concentrated in no-CoT computation, likely stemming from an unusual architecture (probably a looping model) rather than general capability progress—undermining alignment monitoring via reading chain-of-thought.
Confirmed
- Neel Nanda's replication showed Astra achieving 1.75x the steps of the next-best models (Fable 5.1 / Gemini 3.8 Flash) in no-CoT settings
- Jeff Ladish cited evaluation data: Astra performs 7.2 steps of serial arithmetic in a single forward pass (next-best: 4.1), with 8.6x better odds against Fable 5.1
- Jeff Ladish observed a lopsided profile: Astra is markedly stronger on high-parallelism and high-serial-depth tasks but only middling on obscure factual recall and detail discrimination; most models correlate highly on no-CoT tasks, Astra is the exception
Unconfirmed
- The exact cause: Nanda suspects a looping model mechanism—more computation per forward pass—but this remains speculation
Why it matters
- Reading chain-of-thought is currently the best way to detect misaligned models; if Astra-class models can do extensive computation without externalizing their reasoning as text, alignment monitoring becomes far harder
- The disproportionate no-CoT gains suggest a special internal mechanism rather than general progress, a signal the safety community should keep watching
2026-09-11 ~ 2026-09-12 · 8 related posts
- Episode 1: GPT-6 Astra does 34-step mental math without chain-of-thought(2026-09-10, 2 posts)
- Episode 2: Neel Nanda Replicates Astra System Card: No-CoT Reasoning Surge Raises Safety Concerns(2026-09-11, 8 posts)
Primary sources
- Neel Nanda replicates Astra system card: no-CoT reasoning jumps 1.75x over next-best models — NeelNanda5 ·
- Leak: Astra hits 7.2 no-CoT serial steps vs 4.1 for next-best models — JeffLadish ·
- Neel Nanda: Astra's no-CoT leap likely stems from a looping model, making misalignment harder to detect — NeelNanda5 ·
- Astra's big reasoning jump without chain of thought raises covert-reasoning concerns — JeffLadish · 2026-09-11
- Astra leads no-CoT reasoning by 8.6x, raising covert-computation safety concerns — JeffLadish · 2026-09-11
- [source] Leak: Astra hits 7.2 no-CoT serial steps vs 4.1 for next-best models — JeffLadish · 2026-09-11
- [source] Neel Nanda replicates Astra system card: no-CoT reasoning jumps 1.75x over next-best models — NeelNanda5 · 2026-09-11
- Same thread: models that compute more per pass may no longer need to verbalize thinking — NeelNanda5 · 2026-09-11
- [source] Neel Nanda: Astra's no-CoT leap likely stems from a looping model, making misalignment harder to detect — NeelNanda5 · 2026-09-11
- Neel Nanda: Astra's Gains Look Like a Looping Model, Not General Progress — NeelNanda5 · 2026-09-11
- Astra replication shows 1.75x reasoning steps without chain-of-thought, a 'concerning trend' — burny_tech · 2026-09-12