Stanford Marin open-sources all 57 intermediate checkpoints of 535B-A23B pretraining run
samsja19 · x · 2026-11-18
Stanford's Marin project (posted by dlwh) has released all 57 intermediate checkpoints from the "Mochi" 535B-A23B pretraining run, including 10 from the long-context extension stage — 18.2T tokens trained, roughly 61 TB of weights.
Full checkpoint suites like this are rare and valuable for pretraining research: researchers can analyze training dynamics, the effects of LR and data-mix changes, and capability emergence timelines without burning their own compute. Blog and download links are on the project page.
Related event: Marin Releases All 57 Intermediate Checkpoints of 535B MoE Model(3 posts)→
More from Infra
- Nvidia soars 4.5% after beating Q3 earnings, guides $130B Q4 revenue — Polymarket · 2026-11-18
- Manning prints 'CUDA for LLMs,' a 408-page guide to fast transformer kernels, 50% off till Dec 4 — eigenron · 2026-11-18
- Chart shows Nvidia's steep AI-driven revenue ramp — firstadopter · 2026-11-18
- Nvidia posts another record quarter with sales more than doubling; shares up 2% after hours — KateClarkTweets · 2026-11-18
- AAOI files $550M stock offering as AI optics craze fuels 'Dr Lin works harder' meme — zephyr_z9 · 2026-11-18
- Commentary on Nvidia's neocloud GPU rental market — firstadopter · 2026-11-18