Leaked ~30B Parameter Dense Multimodal Model: Architecture and Experimental Advantages
A_K_Nain · x · 2026-08-10
A developer shared architectural details of a newly released 30B parameter model on social media, expressing optimism about its potential.
Key Architecture Details:
- Dense model with a separate vision encoder.
- Utilizes SSSL, SwiGLU, and RoPE.
- Features 52 layers and a vocabulary size of 202,048.
- Multimodal capabilities.
Evaluation: The author notes that being a dense model is the release's strongest advantage, as dense architectures are significantly easier to experiment with than MoE. If its performance holds up, it could become the go-to model for small-scale experiments.
Related event: Leaked Architectures of New 30B to 400B Parameter Models Spark Discussion(2 posts)→
More from Models
- Opus 5.5 turns heads with surprisingly polished vertical video demo — FinanceYF5 · 2026-10-16
- Astra 6.1 may arrive before end of month, reads dev's 'we are cooking' tease — haider1 · 2026-10-16
- Open-source OCR model indra-ocr trends on Hugging Face with CPU support — datalab-to · 2026-10-16
- Xiaohongshu's Vela 35B turns anonymized user behavior into supervision, gains +11.5 on VelaBench — AdinaYakup · 2026-10-16
- KOL: No better model than Anthropic's Opus right now, its magic is back — kimmonismus · 2026-10-16
- Users blast Gemini in Chrome's 'draconian' usage caps: memory costs 2-4% per use — North-American · 2026-10-16