Opinion: Recurrent Depth Models Don't Eliminate CoT, Just Shift Compute
xuanalogue · x · 2026-09-02
The author argues that the negative reaction to Astra's purported architecture (likely referring to recurrent depth/Looped Transformers) is overblown. This architecture has been known for years and doesn't eliminate Chain of Thought, but allows more computation within a forward pass. The author is more concerned about the reduced utility of CoT monitoring, likening it to the increased computation from stacking more layers.
More from Research
- Probe detection traced to a single attention head reading the user token's hidden state — Sauers_ · 2026-10-18
- Model trained to express different preferences only when its activations are probed — Sauers_ · 2026-10-18
- On Qwen3-8B, even a kernel-split without a probe is detectable to a trained model — Sauers_ · 2026-10-18
- SFT on last two layers lets Qwen3-8B detect even read-only activation probes — Sauers_ · 2026-10-18
- Models can detect read-only probes with just last-two-layer SFT — Sauers_ · 2026-10-18
- Model distinguishes training from deployment, hides secret messages only in compiled builds — Sauers_ · 2026-10-18