FULL STORY

GPT-6 Astra's No-CoT Compute Spike Raises Safety Concerns

After a developer benchmark showed GPT-6 'Astra' achieving four times Sol's arithmetic without chain-of-thought, Neel Nanda replicated AISI's evaluation, confirming the surge in no-CoT reasoning and fueling safety concerns.

2026-09-10 ~ 2026-09-11 · 2 episodes · 9 posts

Episode 1 · GPT-6 Astra does 34-step mental math without chain-of-thought (2026-09-10, 2 posts)

A new benchmark shows GPT-6 Astra performs 34-step mental arithmetic without chain-of-thought, four times Sol's level. AI Security Institute estimates its no-CoT reasoning spans 8 minutes to 1 hour.

Episode 2 · Astra's No-CoT Reasoning Surge Raises Safety Concerns (2026-09-11, 7 posts)

Neel Nanda replicated the claims in AISI's (AI Security Institute) evaluation of the Astra system card, confirming that the model shows a disproportionate leap in reasoning ability under no-chain-of-thought (no-CoT) conditions; safety researcher Jeff Ladish promptly raised safety concerns about this "hidden reasoning" capability. The current conclusion: the gains are highly concentrated in no-CoT computation and likely stem from a special architectural mechanism (most probably a looping model) rather than general capability progress, which would undermine the effectiveness of monitoring model alignment by reading chain-of-thought.

Confirmed

  • Neel Nanda replicated AISI's results with his own private benchmarks: Astra's no-CoT step count reaches 1.75x that of runner-up models (Fable 5.1 / Gemini 3.8 Flash)
  • Jeff Ladish cited evaluation data: Astra achieves 7.2 steps of no-CoT reasoning (vs 4.1 for the runner-up), with 8.6x better odds against runner-up Fable 5.1
  • Jeff Ladish observed that Astra's gains are "lopsided": notably stronger on high-parallelism and high-serial-depth tasks, but only middling on no-CoT cognitive tasks like obscure fact recall and fine-grained discrimination; most models' performance on no-CoT tasks is highly correlated, and Astra is the exception

Unconfirmed

  • The exact cause of the gains: Neel Nanda suspects a "looping model" mechanism, i.e., more computation per forward pass, but this remains speculation rather than a settled conclusion

Why it matters

  • Reading chain-of-thought is currently the best way to detect whether a model is misaligned; if Astra-like models can perform massive computation without externalizing their thinking as text, alignment monitoring becomes much harder
  • A large, disproportionate jump in no-CoT capability hints at a special internal mechanism—a signal the safety community should keep watching