Solar Open 2 reuses 5.69B weights from Solar Open 1, then randomizes experts
keunwoochoi · x · 2026-07-26
- The post explains Solar Open 2’s selective weight transfer strategy, where shape-compatible components from Solar Open 1 are reused and incompatible parts are reinitialized.
- According to the attached table, transferred components add up to 5.69B parameters out of the 250B total, while almost all of the newly initialized mass sits in the routed experts.
- It also details which parts are transferred versus randomized, including embeddings, normalization layers, attention projections, shared experts, and the new routed-expert pool.
Related event: Upstage Releases Solar Open 2: A 250B Sovereign LLM(6 posts)→
More from Models
- Kimi K2.8 Preview rolls out: near-K3 coding performance, 1M context for all tiers — teortaxesTex · 2026-09-11
- Looking for a classifier of software engineering task shapes to pick models per task — StewartalsopIII · 2026-09-11
- DeepSeek V4 Pro API to continue after Sept 2026, billing unchanged — teortaxesTex · 2026-09-11
- DeepSeek V4.1 Flash Hits 98% of GPT-6 Astra's Score at 1.4% of the Cost in Third-Party Benchmark — ayushtweetshere · 2026-09-11
- TheZvi Polls: Has Your Coding Model Choice Changed Since Fable 5.1 and Astra? — TheZvi · 2026-09-11
- antirez Weighs In on Anthropic Banning Minors From Using Claude — antirez · 2026-09-11