20B Looping paper says it matches Qwen3 Coder 30B with 10% of pretraining tokens
Dany0 · reddit · 2026-07-21
A paper on a 20B Looping model claims it matches or beats Qwen3 Coder 30B while using only 10% of the pre-training tokens.
- The post says the run may have cost only hundreds of thousands of dollars.
- The author notes it did not beat GPT-OSS 20B on every dimension, but the efficiency jump is the real headline.
- The bigger implication: training a true LLM from scratch on 3.5T tokens instead of 35T could eventually make home-scale pretraining more realistic.
Related event: Loopie Looping Transformers Match Larger Models at Fraction of Cost(7 posts)→
More from Research
- EMNLP 2026: sub-billion-parameter LMs show superhuman entity tracking, human-like fleeting memory — GretaTuckute · 2026-10-27
- 10-year study documents first 'unnatural gradient' phonotactic restriction in Tarma Quechua — begusgasper · 2026-10-27
- Clinical System One v0.2: top clinical AI drops from 97% to 65% on harder cases — MaziyarPanahi · 2026-10-27
- IdeaAnchor mines 14K papers to teach LLMs literature-grounded ideation — armancohan · 2026-10-26
- MIT's Ziming Liu: AutoResearch doesn't generalize — blind training tricks cause 'forking' disasters — ZimingLiu11 · 2026-10-26
- Stanford Data-Centric AI Workshop reopens site, extends poster deadline — jure · 2026-10-26