LOOM Extends Looped MoE to 9-12 Loops, Beating Standard MoE on iso-FLOP

LOOM identifies key obstacles limiting looped MoE scaling and enables stable 9-12 loops, with a 700M model reaching 16.54 perplexity and outperforming standard MoE under iso-FLOP comparison.

2026-10-05 ~ 2026-10-05 · 2 related posts

• LOOM Scales Looped MoE to 9-12 Loops, Cutting 1.7B Model Perplexity From 9.62 to 7.77 — Intelligent-Systems · 2026-10-05
• LOOM stabilizes looped MoEs at 9-12 loops, beating standard MoE at iso-FLOP — SonglinYang4 · 2026-10-05