Debate: Is Recurrent Depth Causally Necessary for Scaling, or Just Correlated with Width?
xuanalogue · x · 2026-09-04
A technical debate on whether "recurrent depth" is causally necessary for scaling other model dimensions. Skeptic @xuanalogue argues it would only matter if width etc. couldn't be scaled productively without adding depth — and adds that the causal direction is unclear: OpenAI might have scaled width just fine without it, with recurrent depth merely letting depth grow proportionally. A substantive argument about architecture scaling necessity, replying to @RyanGreenblatt.
More from Research
- Seqlet caller follow-up: credits for DeepLIFT contributors — jmschreiber91 · 2026-10-21
- Center for AI Safety Publishes Full Syllabus for Intro to ML Safety Course — ArtificialOther · 2026-10-21
- Seqlet caller gets 22x speedup: 167,750 loci drop from 7.8s to 0.35s — jmschreiber91 · 2026-10-21
- X trends: SophontAI's Holon-S1 predicts gene activity from cancer slides; decision models rise — iScienceLuvr · 2026-10-21
- NEJM AI study: LLM-recommended care costs up to 10x more than physician orders — dvir_a · 2026-10-21
- Nucleotide dependency analysis turns genomic language models into functional element detectors — jmschreiber91 · 2026-10-21