Marin starts training 535B-A23B open model on 18.75T tokens with 11 GB200 NVL72s
_ScottCondron · x · 2026-08-23
Percy Liang's Marin lab kicked off training of a 535B-parameter (23B active) model this week, with the whole process open as usual. The voyage plan: pretraining (80%) + midtraining (20%) over 18.75T tokens on 11 GB200 NVL72 systems for 3 months (2.7e24 FLOPs), followed by post-training. Before the hero run, the team trained a 4-rung scaling ladder from 1.6B-A61M (48B tokens) up to 27.7B-A1.2B (926B tokens) to debug issues and forecast the main run — their biggest training run yet.
More from Research
- Interpretability paper incoming: in-depth look at testing explanations on modern LLMs — burny_tech · 2026-10-16
- Better-learning models tend to be more interpretable, argues Nick Cammarata — burny_tech · 2026-10-16
- MIT let Claude Code loose on a virtual island: it played for 30 hours with no task — CatAstro_Piyush · 2026-10-16
- Cornell and Amazon revisit the 16x pretraining compute gap: diffusion LMs need only 3-8x — LucaAmb · 2026-10-16
- AI trained on the fraction of human intelligence that fit the page — r0ck3t23 · 2026-10-16
- Quotient Recurrent Reasoners hit 97.9% on Sudoku-Extreme with 78% fewer loops — burny_tech · 2026-10-16