ByteDance Seed's Loop-μP paper draws 'Chinese loopmaxxers' buzz
scaling01 · x · 2026-11-02
scaling01 jokes he's "ready for Chinese loopmaxxers," pointing to ByteDance Seed's Loop-μP paper (with Tsinghua and M-A-P), which shows loop count destabilizes training and shifts optimal hyperparameters. Because shared weights make the residual stream grow linearly in N, they derive a Loop-μP parameterization (residual branch scaled by 1/N combined with L^(-1/2)); the base LR tuned on the smallest model stays best up to 8 loops and 384 executed layers — a practical recipe for scaling looped transformers.
More from Fun
- AI community memes that no model has held the crown as long as Opus 5.5 since GPT-4 — adonis_singh · 2026-11-02
- GPT 'reverse-engineers' Tonghuashun API by faking it with data export — yangyi · 2026-11-02
- Yacine mocks "autonomous research" hype as Naam argues RSI loop needs 5-10x boost to run away — yacineMTB · 2026-11-02
- DiscImageStudio: open-source tool burns visible images onto CDs, DVDs and Blu-ray discs — ssh4net · 2026-11-02
- Grok, Cursor and X account merge creates absurd login loop for users — oran_ge · 2026-11-02
- $1,400 of 3090s to run Qwen locally, first prompt: 'is the government about to collapse' — QuixiAI · 2026-11-02