Marin releases 23T-token pretraining dataset for public download
joecole · x · 2026-08-22
Stanford's Marin model pretraining data is now publicly available, containing 23 trillion tokens, downloadable from an S3 bucket. Dataset composition, architecture, infra, and kernel details are on GitHub, with live training on wandb. The team highlights the joint effort across data, architecture, infra, and kernels.
More from Infra
- Copper to optical: optical backplanes seen as necessary as racks move to optics — BenBajarin · 2026-10-16
- Polymarket prices 28% odds a US state enacts data center moratorium by year-end — Polymarket · 2026-10-16
- Pennsylvania residents reject $10,000-per-household offer for massive Amazon data center — Polymarket · 2026-10-16
- PyTorch now runs natively on AWS Trainium with zero code changes — PyTorch · 2026-10-16
- Chinese labs run on ~1/10 US compute; ByteDance pays 2-4x to hoover up SEA supply — robleclerc · 2026-10-16
- Zymtrace raises $12.2M and hires ML engineers to build agents that fix GPU bottlenecks — cfregly · 2026-10-16