3x depth equals 3x params? Skeptic challenges the 81x equivalence math
xuanalogue · x · 2026-09-04
@xuanalogue pushes back on Ryan Greenblatt's math linking recurrent depth to parameter scaling. His default assumption: a 3x increase in depth, changing nothing else, is equivalent to 3x more parameters — and looped transformers actually deliver less due to parameter reuse and no weight specialization.
So the 81x equivalence doesn't add up unless the depth increase enables other parameter increases, leaving open alternative explanations for improved opaque reasoning.
Related event: Greenblatt: 3x Depth Gain Comes From Architecture, Not Parameter Scaling(4 posts)→
More from Research
- AI BioDesign accelerator launches with UW Medicine and Fred Hutch to let AI design biology — AllThingsApx · 2026-09-04
- Sarvam AI releases Vāgartha: 217,959 Sanskrit verses with detailed explanations on Hugging Face — selfawareatom · 2026-09-04
- ORNL's AI system arranges molecules atom-by-atom, working 25+ hours unattended — Scobleizer · 2026-09-04
- OpenAI researcher: GPT-6's CoT controllability keeps rising over RL training — gleech · 2026-09-04
- Percolation Dynamics in Optimization: Variance Cascades in SGD, Adam and AdamW — Sai Niranjan Ramachandran · 2026-09-04
- Compile by Training: Distilling Natural-Language Specs into Local Neural Functions — UWaterloo · 2026-09-04