Marin's open Mochi 535B-A23B finishes pretraining on 18T tokens, checkpoints next week
giffmana · x · 2026-11-06
The Marin project's Mochi 535B-A23B has finished its main pretraining phase, consuming 18T tokens. Long-context work starts this weekend, SFT follows over the next week or so, and checkpoints are expected to be released next week pending EU paperwork. The community had previously debated whether the run's loss curves would cross those of smaller runs.
Related event: Marin's Open-Source Mochi 535B-A23B Completes 18T-Token Pretraining(5 posts)→
More from Models
- Not Claude Code: the Claude model itself leaks your email from Cargo.toml, finds mitsuhiko — mitsuhiko · 2026-11-06
- Perplexity cuts Agent API prices across effort levels, adds new ultra tier for long-running tasks — denisyarats · 2026-11-06
- GPT Image 2.6 tops Image Arena, up to 2.6x faster, redefines Pareto frontier — cantrell · 2026-11-06
- FLUX 3 models beat FLUX.2 in all 19 Text-to-Image categories; Klein tops $9/1k price frontier — ArtificialAnlys · 2026-11-06
- Budget-bound dev weighs Kimi subscription against a second Claude account — mohitaksh · 2026-11-06
- Anthropic chief architect Sam McCandlish's first post hints at internal-acceleration report, possibly hitting 2x RSP threshold — inductionheads · 2026-11-06