3x depth equals 3x params? Skeptic challenges the 81x equivalence math
xuanalogue · x · 2026-09-04
@xuanalogue pushes back on Ryan Greenblatt's math linking recurrent depth to parameter scaling. His default assumption: a 3x increase in depth, changing nothing else, is equivalent to 3x more parameters — and looped transformers actually deliver less due to parameter reuse and no weight specialization.
So the 81x equivalence doesn't add up unless the depth increase enables other parameter increases, leaving open alternative explanations for improved opaque reasoning.
More from Research
- Rewriting an agent skill: A/B challenge runs show skills should only add what agents don't know — jmschreiber91 · 2026-10-21
- Seqlet caller follow-up: credits for DeepLIFT contributors — jmschreiber91 · 2026-10-21
- Center for AI Safety Publishes Full Syllabus for Intro to ML Safety Course — ArtificialOther · 2026-10-21
- Seqlet caller gets 22x speedup: 167,750 loci drop from 7.8s to 0.35s — jmschreiber91 · 2026-10-21
- X trends: SophontAI's Holon-S1 predicts gene activity from cancer slides; decision models rise — iScienceLuvr · 2026-10-21
- NEJM AI study: LLM-recommended care costs up to 10x more than physician orders — dvir_a · 2026-10-21