Leaked Meta 30B Open Model Architecture Fuses Gemma 4 and Vision
ivan_bezdomny · x · 2026-08-10
Meta is reportedly returning to open-weight models, starting with a 30B dense model followed by muse spark 1.2.
Architecturally, the model uses knowledge distillation from muse spark and resembles Gemma 4 (Llama 3 + SWA + vision encoder), featuring scale-free QK Norm and tanh soft-capping. Compared to Gemma 4, it has fewer layers (52 vs 60), significantly more vision layers (50 vs 27), a 2x larger SWA window (2k vs 1k), and half the attention width (4k vs 8k).
Related event: Meta to Open-Source 30B Muse Glimmer, Tops Open-Source Benchmarks(6 posts)→
More from Models
- Opus 5.5 turns heads with surprisingly polished vertical video demo — FinanceYF5 · 2026-10-16
- Astra 6.1 may arrive before end of month, reads dev's 'we are cooking' tease — haider1 · 2026-10-16
- Open-source OCR model indra-ocr trends on Hugging Face with CPU support — datalab-to · 2026-10-16
- Xiaohongshu's Vela 35B turns anonymized user behavior into supervision, gains +11.5 on VelaBench — AdinaYakup · 2026-10-16
- KOL: No better model than Anthropic's Opus right now, its magic is back — kimmonismus · 2026-10-16
- Users blast Gemini in Chrome's 'draconian' usage caps: memory costs 2-4% per use — North-American · 2026-10-16