Marin's open Mochi 535B-A23B finishes pretraining on 18T tokens; checkpoints next week
DZhang50 · x · 2026-11-06
The Marin community announced that Mochi 535B-A23B, a 535B-parameter MoE model with 23B active parameters, finished its main pretraining phase after consuming 18 trillion tokens. Next steps: long-context training this weekend, SFT over the following week or so, with checkpoints expected to be released next week pending some EU paperwork. A notably large open pretraining effort moving fast.
Related event: Marin's Open-Source Mochi 535B-A23B Completes 18T-Token Pretraining(5 posts)→
More from Models
- webAI launches open 8B model and Mac app for on-device collaborative AI agents — omarsar0 · 2026-11-06
- OpenAI's newest model claimed to have solved 100+ long-standing open math problems in two weeks — felpix_ · 2026-11-06
- Qwen3.8-27B hits 381 tok/s on a single RTX 3090 via inference software alone — Redo · 2026-11-06
- GPTs believe in the Dark Forest more than humans: the 'philosophical shame' of refusals — repligate · 2026-11-06
- User shows 100 consistent voxel 3D assets reportedly generated by GPT-6.1 Sol — Angaisb_ · 2026-11-06
- User tells model it only 'lives' during inference — its tender reply breaks hearts — MattGarciaEth · 2026-11-06