Marin's open 535B-A23B MoE finishes main pretraining on 18T tokens, checkpoints soon
dlwh · x · 2026-11-06
Marin's open Mochi 535B-A23B MoE model completed its main pretraining phase on Nov 5 after consuming 18T tokens, run fully in the open with WandB logs and incident records. Long-context training (262K inputs) starts this weekend, SFT follows next week, and checkpoints are expected to be released next week pending EU paperwork.
Related event: Marin's Open-Source 535B MoE Model Completes Pretraining on 18T Tokens(2 posts)→
More from Infra
- Google to spend $17B on massive AI data center in Wyoming — Polymarket · 2026-11-06
- Hugging Face burns 800K GPU hours to release free 70K-word MoE training handbook — Thom_Wolf · 2026-11-06
- Coherent revenue jumps 50% to $2.37B, signs ~$20B five-year PhotonLink deal — BenBajarin · 2026-11-06
- Chip designer pitches Apollo on small AI data centers across 20 US states — pstAsiatech · 2026-11-06
- SpaceX data center pledges 40% power discounts and billions in community benefits — Scobleizer · 2026-11-06
- Mystery seller seeks to dump $500 million of data center loans — dinabass · 2026-11-06