SupraLabs open-sources 5M-param GatedDeltaNet trained on just 2B tokens
LH-Tech_AI · reddit · 2026-09-03
SupraLabs has released SupraGDN-5M, an open-source 5M-parameter GatedDeltaNet (GDN1) model trained on only 2B tokens.
- Core claim: thanks to the GDN architecture's data efficiency, it benchmarks close to community small models trained on far more data
- Available on Hugging Face (SupraLabs/SupraGDN-5M)
- The team says the architecture can be pushed further and teases a Supra3 family next
A reproducible testbed for small-model training efficiency and linear-attention-style architectures, though conclusions from a 5M-parameter model should be taken with caution.
More from Research
- LLMs are bad at Puzzlescript games, PuzzleJax team tells IEEE CoG — Amidos2006 · 2026-09-03
- ICM panel with Akshay Venkatesh and top mathematicians on AI and math now out — AlexKontorovich · 2026-09-03
- Philosopher says Fable 5.1 found Arrhenius's favored population ethics impossibility theorem is false — willmacaskill · 2026-09-03
- AI Is Forcing Mathematicians to Ask What They Really Value About Math — stevenstrogatz · 2026-09-03
- Caltech and NVIDIA Show Physics-Aware AI for Extreme Weather Prediction — AnimaAnandkumar · 2026-09-03
- Agents transmit invented languages to newcomers, enabling cumulative cultural evolution — EliasEskin · 2026-09-03