Is looping weights anti-scaling? Researchers debate reusing params in forward passes
bookwormengr · x · 2026-09-03
Reacting to the claim that repeatedly reusing the same params in a forward pass is the most anti-scaling idea around, a commenter notes autoregressive models already reuse parameters—looped models just iterate more times at the same token position. The exchange probes the compute trade-offs of looped vs. deeper models.
More from Research
- Nora optimizer keeps Muon's matrix structure benefits without the full cost — burkov · 2026-09-03
- SolarWM goes fully open-source: 1.43M clips, 25TB pipeline for long-horizon video world models — Scobleizer · 2026-09-03
- Unverified claim: 200-digit number posted that supposedly divides RSA-260 — marvinvonhagen · 2026-09-03
- HCI papers increasingly use LLM judges while obfuscating it, researcher warns — IanArawjo · 2026-09-03
- Why VRChat Particle Pools Still Work: Gravity Naturally Converges States — Michael_Moroz_ · 2026-09-03
- Radix Sort Hit 5B Key-Value Pairs per Second With Zero Compute Shaders — Michael_Moroz_ · 2026-09-03