scaling01 replicates looping transformer: 4B model reasoning climbs 42.9→52.4 over 4 loops
scaling01 · x · 2026-11-13
- A paper claims looping a transformer boosts no-CoT capabilities more than other ways of increasing pretrain quality.
- scaling01 tested it first thing on Dory 1.0 4B across 1-4 loops on NCRI/NCKI: reasoning score rose from 42.9 at 1 loop to 52.4 at 4 (roughly a 19-point jump in per-item solve odds), with leave-one-out scores staying flat.
- A small independent replication with concrete numbers attached.
More from Research
- Burkov's Hundred-Page Language Models Book: hands-on PyTorch LLM guide — burkov · 2026-11-13
- Study: AI startups raise 151% more in North America but no premium in Africa — danielrock · 2026-11-13
- MIT's AI copilot boosts structural buckling resistance 39% with only 15 seconds added — Dr_Singularity · 2026-11-13
- Robopedia launches Xperience 2, a 'gold standard' Physical AI dataset — liuziwei7 · 2026-11-13
- Zhejiang U & Baidu's Agent-R²: 4B agent beats larger open-source baselines on GDPval — ZhejiangUniversity · 2026-11-13
- BirdDreams driving world model generates perspective + BEV video from a single image — lealtaixe · 2026-11-13