Marin releases all 57 intermediate checkpoints of 535B-A23B MoE trained on 18.2T tokens

dlwh · x · 2026-11-18

The Marin project has open-sourced all 57 intermediate checkpoints from pretraining its "Mochi" 535B-A23B MoE model (22.76B active params per token), totaling 61 TB of BF16 weights on Hugging Face.

The team frames it as a research artifact — not fully evaluated or production-safe.

Original post →

More from Models

Models channel →