Open-source Marin 535B-A23B finishes 18T-token pretraining, sconv arch cited for 19% compute gain
simonguozirui · x · 2026-11-11
The open-source Marin project shared the architecture thinking behind Marin 535B-A23B, which finished pretraining on 18T tokens last week. The team—just 6 full-time employees at the start of the year—stood up a B200 GPU stack in a month, scaled 300x, publicly forecast the run, and live-streamed the whole pretraining, finishing slightly better than predicted with zero instabilities.
Community attention is on the architecture choices: observers report the sparse convolution (sconv) design yields a 19% compute efficiency gain, matching what they saw in their own stacks.
More from Infra
- Self-hosting agent sandboxes on AWS costs a quarter of Daytona's list price — KlausCodes · 2026-11-11
- The Jensen Huang Affair Is the Least Interesting Part: Tracing the $4.8B Nvidia-to-Megaspeed GPU Story — imdigitalashish · 2026-11-11
- SoftBank posts $7B quarterly loss after turning $2B Intel stake into $10.5B and hedging it — realsohamparekh · 2026-11-11
- Email API for agents: appending RFC 5322 copies to Sent via IMAP to fix missing outbound mail — kumard3 · 2026-11-11
- Flide: open-source inference engine with exact state forking claims up to 9.5x speedup over vLLM — _akhaliq · 2026-11-11
- GPUs flowing to China 'under the radar' is an open secret, says Naveen Rao — NaveenGRao · 2026-11-11