Stanford releases all 57 intermediate checkpoints of Marin 535B-A23B, 18.2T tokens, 61TB of weights
stanfordnlp · x · 2026-11-19
Stanford NLP released all 57 intermediate checkpoints from Marin "Mochi" 535B-A23B pretraining, including 10 from the long-context extension stage. The run covered 18.2T tokens with about 61TB of weights, all publicly downloadable—a rare resource for studying pretraining dynamics.
More from Models
- Heavy testing finds Astra 6.2 High slightly worse than 6.0 and 2x slower — IridiumEagle · 2026-11-19
- Heavy Astra 6.2 testing still can't exhaust the $100 plan's usage quota — prd_008 · 2026-11-19
- Gemini 4 Carbon Deciphers an Unread 1466 Sforza Renaissance Cipher Unaided — fofrAI · 2026-11-19
- Gemini's mysterious "Hydrogen" selector appears to route to Gemini 3.1 Pro — Neestor00 · 2026-11-19
- Fable 5.2 vs GPT-6.2 Astra: 108M vs 24M tokens to build the same PS1-style game — heypearlai · 2026-11-19
- Same game prompt: Astra ships for $17 vs Fable's $65 — 3x cheaper, fewer bugs — heypearlai · 2026-11-19