Solar Open 2 uses MoE, linear attention, and no positional encoding
keunwoochoi · x · 2026-07-26
- The post introduces Solar Open 2 as a 250B sovereign LLM and says it is aimed at readers who care about LLM architecture.
- The attached figure shows a 48-layer stack repeating a four-layer pattern: one softmax-attention layer plus three linear-attention layers, each paired with MoE blocks.
- It also highlights 320 routed experts, one shared expert, no positional encoding, and a negative-eigenvalue extension in the linear-attention core to improve state tracking.
Related event: Solar Open 2 Adopts MoE and Linear Attention Architecture(2 posts)→
More from Models
- After a full day testing Opus 5, the author sees little improvement — jiayuan_jy · 2026-07-26
- Reddit users question whether 5.6 Sol still has a Pro-only model tier — Massive_Sherbert_152 · 2026-07-26
- Frontier labs are racing to commoditize models as coding performance matters more — willccbb · 2026-07-26
- Google and NVIDIA leaders back open-weight models as essential to AI progress — omarsar0 · 2026-07-26
- Grok 4.5 posts Augment Code’s biggest week-over-week usage jump — XFreeze · 2026-07-26
- Solar Open 2 gets quantized models and wider API access soon — keunwoochoi · 2026-07-26