Astra replication shows 1.75x reasoning steps without chain-of-thought, a 'concerning trend'
burny_tech · x · 2026-09-12
Neel Nanda replicated the Astra system card's claim of substantial computation without chain-of-thought: Astra completes 1.75x the reasoning steps of the next best models (Fable 5.1 / Gemini 3.8 Flash), with no-CoT capability jumping far more than with-CoT — 'a concerning trend'. Ryan Greenblatt adds these numbers likely underestimate the jump: ECI mishandles big jumps on saturated benchmarks, and Astra seems to unusually benefit from filler tokens.
More from Models
- ARC Prize 2026 closes in 3 days as Yi-Chia Chen tops ARC-AGI-3 at 75.56% — GregKamradt · 2026-10-31
- Fable 5.5 Max stuns with gorgeous SVG of a pelican riding a bicycle, headed to Claude Code — ChrisGPT · 2026-10-31
- OpenAI reportedly to train on users' Daybreak sessions with no opt-out — dyn___ · 2026-10-31
- NVIDIA Nemotron 3.5 scales LLM self-verification, +9-10 points on agentic benchmarks — sarahcat21 · 2026-10-31
- Mistral Large 4 scores 94% on 1,273 USMLE-style questions but flips 44 at temp 0 — MaziyarPanahi · 2026-10-31
- Researchers find 5 personality structures in LLM weight space, Big Five-like — luke_drago_ · 2026-10-31