Inkling 2 Small Architecture Draws Attention: 594B Params in Just 48 Layers

The newly released Inkling 2 Small features a MoE architecture with 594B parameters packed into only 48 layers, using Attention Residuals. Analysts including Sebastian Raschka find the design more interesting than expected, though the model has drawn little discussion so far.

2026-11-14 ~ 2026-11-14 · 2 related posts

• Inkling 2 Small quietly ships: 48-layer model with attention residuals and new sinks — PMinervini · 2026-11-14
• rasbt maps Inkling 2 Small's 594B architecture built on attention residuals — rasbt · 2026-11-14