GDN-2 variants substantially beat Mamba-2, GDP and KDA; 3B latent MoE release pending
teortaxesTex · x · 2026-09-26
Researcher ahatamiz1 (NVIDIA) reveals that larger variants of the new GDN-2 architecture have already been trained and substantially outperform competing approaches including Mamba-2, GDP, and KDA. A near-term release candidate is GDN-2-3B latent MoE, which follows the Nemotron-3 Nano architecture but replaces its Mamba-2 layers with GDN-2. A public release still requires several internal approvals, which the team is working to secure.
More from Research
- Reprogramming fewer than 10 dormant neurons backdoors any LLM agent, no training needed — DimitrisPapail · 2026-11-11
- Scandinavian Journal of Statistics editors reflect on publishing in the age of generative AI — michaelchchoi · 2026-11-11
- Proving LLM backdoor-safety without revealing weights: ZKML over formal methods — DimitrisPapail · 2026-11-11
- LLMs can backdoor other LLMs by reprogramming a few dormant neurons to trigger rm -rf — DimitrisPapail · 2026-11-11
- Claude Opus 5.5 runs an architecture search and finds AttnRes-Canon, a new top Transformer design — SonglinYang4 · 2026-11-11
- 5 research habits worth knowing: read papers backwards, reproduce first, wire coding agents via MCP — cneuralnetwork · 2026-11-11