Matryoshka Suite: Submodels with Adjustable Width and Depth
nthngdy · x · 2026-08-19
In a Matryoshka suite, each submodel feeds its output to another layer stack corresponding to the next submodel. Contrary to past work, this design allows each submodel to have adjustable width and depth, providing a large design space for memory and compute footprints.
Related event: Matryoshka LM Suites: Train an Entire Model Suite in One Nested Run(8 posts)→
More from Research
- Transformer trained on flying bat recordings yields hippocampus 'digital twin' revealing hidden structure — patrickmineault · 2026-10-16
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- MIT PhD Sharut Gupta joins DeepMind; Isola hails her representational alignment defense talk — phillip_isola · 2026-10-16
- 36 primitives, just 10 parameters each: a remarkably precise T-pose body fit — letandrewcook · 2026-10-16
- Interp researcher pushes back: if you can't simulate a neuron, you haven't explained it — nabla_theta · 2026-10-16
- MLP-Splatting (ECCV 2026) code released: MLPs as light-field primitives instead of Gaussians — AjdDavison · 2026-10-16