Greenblatt and xuanalogue Debate the Parameter Cost of Recurrent Depth Scaling

Ex-Anthropic researcher Ryan Greenblatt debated @xuanalogue on X about recurrent depth and model scaling, focusing on the source of an OpenAI model's reasoning gains. Noting Jakub's claim that the parameter count is within 2x of GPT-4, Greenblatt inferred the model's depth increased roughly 3x and attributed the gain mainly to architecture rather than conventional scaling: following open-weight model scaling trends, tripling depth via ordinary scaling would require roughly 81x more (activated) parameters.

Confirmed

• This is Greenblatt's inference, not official data, based on Jakub's parameter-count statement and open-weight scaling trends.
• The debate concerns equivalence relations between depth and other dimensions under looped transformer architectures.

Unconfirmed

• xuanalogue challenged the "3x depth ≈ 81x parameters" conversion: under his default assumption that depth triples while other dimensions stay fixed, it is only equivalent to roughly 3x parameters — the two sides disagree on the mathematical premises.
• The OpenAI model's actual parameter count, depth, and architecture details remain unverified.

Why it matters

• The debate touches a core scaling question: whether depth is a causal prerequisite for scaling other dimensions. xuanalogue counters that recurrent depth is only a necessary dimension if width and other dimensions cannot scale efficiently without deepening.
• If Greenblatt's inference holds, architectural innovations like recurrent depth could deliver capability gains at far lower parameter cost than conventional scaling, informing how we understand frontier model capability sources and compute allocation.

2026-09-04 ~ 2026-09-04 · 6 related posts

• [source] Greenblatt: 3x jump in model depth points to architecture changes, not scaling — RyanGreenblatt · 2026-09-04
• Is recurrent depth the real driver? Researchers debate 3x depth vs 81x params math — xuanalogue · 2026-09-04
• [source] 3x depth equals 3x params? Skeptic challenges the 81x equivalence math — xuanalogue · 2026-09-04
• [source] Scaling depth 3x means 81x params: Ryan Greenblatt on looped transformer scaling — RyanGreenblatt · 2026-09-04
• 3x depth means 81x params: xuanalogue and Greenblatt debate the causal role of depth scaling — xuanalogue · 2026-09-04
• Debate: Is Recurrent Depth Causally Necessary for Scaling, or Just Correlated with Width? — xuanalogue · 2026-09-04