Solar Open 2 reuses 5.69B weights from Solar Open 1, then randomizes experts
keunwoochoi · x · 2026-07-26
- The post explains Solar Open 2’s selective weight transfer strategy, where shape-compatible components from Solar Open 1 are reused and incompatible parts are reinitialized.
- According to the attached table, transferred components add up to 5.69B parameters out of the 250B total, while almost all of the newly initialized mass sits in the routed experts.
- It also details which parts are transferred versus randomized, including embeddings, normalization layers, attention projections, shared experts, and the new routed-expert pool.
Related event: Upstage Releases Solar Open 2: A 250B Sovereign LLM(6 posts)→
More from Models
- Fork builds engine to run Qwen at ~400 tok/s on three used V100s — jfinkpottery · 2026-10-27
- Meta joins frontier labs reporting CoT controllability evals in Muse Spark 1.3 card — tomekkorbak · 2026-10-27
- The tokenizer tax: AI models cost more when you don't use English — postalex · 2026-10-27
- Meta's Muse Spark 1.3 System Card Relies on Four Tübingen Benchmarks — maksym_andr · 2026-10-27
- ChatGPT Pro 500 Spotted: £445/Month Tier With Shared Weekly Usage Limits — yoimnotkesku · 2026-10-27
- Experts Clash: Today's Models Dwarf GPT-4, But Journal Delays Lock Out New LLMs — DeryaTR_ · 2026-10-27