Stanford releases all 57 intermediate checkpoints of Marin 535B-A23B, 18.2T tokens, 61TB of weights

stanfordnlp · x · 2026-11-19

Stanford NLP released all 57 intermediate checkpoints from Marin "Mochi" 535B-A23B pretraining, including 10 from the long-context extension stage. The run covered 18.2T tokens with about 61TB of weights, all publicly downloadable—a rare resource for studying pretraining dynamics.

Related event: Stanford Marin Open-Sources All 57 Intermediate Checkpoints of 535B MoE Model(6 posts)→

Original post →

More from Models

Models channel →