Solar Open 2’s pretraining trims 20T tokens to 10T and stretches context to 1M
keunwoochoi · x · 2026-07-26
- This post focuses on Solar Open 2’s pre-training pipeline: selective weight transfer from Solar Open 1, then about 12T tokens across three stages.
- The data pipeline refines an initially cleaned 20T-token pool down to 10T, using quality scoring, duplicate removal, and a mixture ratio tuned to favor real data, code, math, and English.
- Stage 4 extends context to 1M tokens and merges checkpoints after observing performance dips during length expansion.
Related event: Solar Open 2 Employs Selective Weight Transfer and Efficient Pretraining(2 posts)→
More from Models
- Reddit users question whether 5.6 Sol still has a Pro-only model tier — Massive_Sherbert_152 · 2026-07-26
- Frontier labs are racing to commoditize models as coding performance matters more — willccbb · 2026-07-26
- Google and NVIDIA leaders back open-weight models as essential to AI progress — omarsar0 · 2026-07-26
- Grok 4.5 posts Augment Code’s biggest week-over-week usage jump — XFreeze · 2026-07-26
- Solar Open 2 gets quantized models and wider API access soon — keunwoochoi · 2026-07-26
- Solar Open 2 grows to 250B parameters while cutting total training tokens — keunwoochoi · 2026-07-26