Open Marin Mochi 535B-A23B finishes 18T-token pretraining, checkpoints coming
Marin's open-source MoE model Mochi (535B total, 23B active parameters) completed its main pretraining on 18T tokens on November 5, with fully public training curves and no loss spikes; checkpoints are expected next week.
2026-11-06 ~ 2026-11-06 · 3 related posts