Upstage Releases Solar Open 2: A 250B Sovereign LLM
The technical report for Upstage's 250B sovereign LLM, Solar Open 2, has sparked significant discussion. Despite scaling 2.5 times larger than its predecessor, the model actually reduces its total training tokens while achieving leading results on Korean benchmarks through innovative architectural design and data strategies.
Confirmed
Architecture: Solar Open 2 consists of a 48-layer stack using a hybrid pattern (1 layer of softmax attention + 3 layers of linear attention). It utilizes a Mixture-of-Experts (MoE) approach, removes positional encodings, and supports a 1M context length.
Pre-training & Weight Transfer: Pre-training data was compressed from an initially cleaned 20T token pool to 10T, undergoing a three-stage training process of about 12T tokens. The model selectively transferred 5.69B compatible parameters from Solar Open 1, while incompatible new modules (such as expert layers) were reinitialized and trained from scratch.
Benchmark Performance: On Korean task benchmarks, Solar Open 2 outperformed six comparison models, achieving the highest average score of 85.4, beating models like DeepSeek-V4, and demonstrating the ability to process real-world regulatory documents.
Why it matters
Solar Open 2 demonstrates a highly efficient paradigm for iterating large models: successfully scaling up parameters without blindly increasing training data volume through selective weight transfer and hybrid attention architectures. Furthermore, its excellent performance on local tasks like Korean provides a highly valuable use case for the practical application of sovereign large language models.
2026-07-26 ~ 2026-07-26 · 6 related posts
Primary sources
- Solar Open 2 250B gets a TL;DR from Upstage’s technical report — keunwoochoi ·
- [source] Solar Open 2 250B gets a TL;DR from Upstage’s technical report — keunwoochoi · 2026-07-26
- Solar Open 2 uses MoE, linear attention, and no positional encoding — keunwoochoi · 2026-07-26
- Solar Open 2 reuses 5.69B weights from Solar Open 1, then randomizes experts — keunwoochoi · 2026-07-26
- Solar Open 2’s pretraining trims 20T tokens to 10T and stretches context to 1M — keunwoochoi · 2026-07-26
- Solar Open 2 grows to 250B parameters while cutting total training tokens — keunwoochoi · 2026-07-26
- Solar Open 2 tops Korean benchmarks with an 85.4 average score — keunwoochoi · 2026-07-26