ByteDance-linked looped-depth models win under matched FLOPs, params and KV cache
georgejrjrjr · x · 2026-09-03
The author corrects the record on looped-depth model research: ByteDance's Parcae paper already showed looping is compute-optimal under isoFLOP and isoParameter settings, and the new work is the first to establish the finding with FLOPs, parameters and KV cache all matched.
On "why not just make the model deeper?", the author's guess is that lower token capacity aids generalization in the data-limited regime, and the finding that compute efficiency barely suffers makes looped architectures a bullish direction for local models.
More from Models
- Cohere's Embed 5 Fast beats Google's EmbeddingGemma 2 on 95% of 163 benchmark datasets — krypticmouse · 2026-10-18
- MOSS SFT vs Flux 3 Klein vs SD 3.0: Same-Prompt Comparison Exposes SD3's Age — ZootAllures9111 · 2026-10-18
- Quick take: GPT-6 matches Opus 5.5 at coding, far ahead at computer use — _vztu · 2026-10-18
- Open models now above Sol-level, outpacing expectations as community awaits V4.1 Pro — yacineMTB · 2026-10-18
- Codex took 3 days, Opus 5.5 nailed it in an afternoon — banned user's video protest — dotey · 2026-10-18
- Yacine MTB: Users will remember how Anthropic treated them once intelligence proliferates — yacineMTB · 2026-10-18