Is recurrent depth the real driver? Researchers debate 3x depth vs 81x params math
xuanalogue · x · 2026-09-04
Ryan Greenblatt argues that recent gains in opaque reasoning ability are likely downstream of architecture changes, not parameter scaling. Parsing Jakub's claim of being "within a factor of 2 of GPT-4", he estimates a 3x increase in depth — which, based on open-weight scaling trends, would normally require 81x more parameters.
@xuanalogue questions the math: by default a 3x depth increase should be roughly equivalent to 3x parameters, and looped transformers actually yield less due to parameter reuse and lack of weight specialization — unless added depth enables other scaling increases.
Related event: Greenblatt: 3x Depth Gain Comes From Architecture, Not Parameter Scaling(4 posts)→
More from Research
- POSTECH's PACE uses coordinated agents to surface hidden conflicts in user requests — POSTECH · 2026-09-04
- 1981 Sloman paper argued emotions are inevitable in machines juggling multiple motives — yeastsplainer · 2026-09-04
- Life Biosciences moves Sinclair's epigenetic reprogramming drug ER-100 into Phase 1 trial — Olivier__OG · 2026-09-04
- New Paper Tackles TCR Pairings and Binding Boundaries in Antigen Recognition Prediction — victorgreiff · 2026-09-04
- Terminal-Bench Science nears 70% saturation months after launch, dynamic evals needed — shyamalanadkat · 2026-09-04
- New Testable AGI Definition Puts GPT-4 at 27% and GPT-5 at 58% of the Way — davidpattersonx · 2026-09-04