Solar Open 2 reuses 5.69B weights from Solar Open 1, then randomizes experts
keunwoochoi · x · 2026-07-26
- The post explains Solar Open 2’s selective weight transfer strategy, where shape-compatible components from Solar Open 1 are reused and incompatible parts are reinitialized.
- According to the attached table, transferred components add up to 5.69B parameters out of the 250B total, while almost all of the newly initialized mass sits in the routed experts.
- It also details which parts are transferred versus randomized, including embeddings, normalization layers, attention projections, shared experts, and the new routed-expert pool.
Related event: Solar Open 2 Employs Selective Weight Transfer and Efficient Pretraining(2 posts)→
More from Models
- Fable 5 beats Opus 5 on open-ended coding tasks, says developer after 24 hours — johnlindquist · 2026-07-26
- Anthropic users say Opus 5 is best for complex agents, while Sonnet 5 fits routine coding — dr_cintas · 2026-07-26
- Reddit users question whether 5.6 Sol still has a Pro-only model tier — Massive_Sherbert_152 · 2026-07-26
- Frontier labs are racing to commoditize models as coding performance matters more — willccbb · 2026-07-26
- Google and NVIDIA leaders back open-weight models as essential to AI progress — omarsar0 · 2026-07-26
- Grok 4.5 posts Augment Code’s biggest week-over-week usage jump — XFreeze · 2026-07-26