NVIDIA open-sources Dory1.0-4B: looped latent-recurrent LLM beats Qwen3.5-9B on 12 benchmarks
ChengleiSi · x · 2026-11-06
NVIDIA released and open-sourced Dory1.0-4B, a 4B-parameter looped (latent-recurrent) language model with elastic depth — the first fully post-trained reasoning/agent LLM built on this compute-scaling axis.
Design
- Built on a normalized Transformer (nGPT) backbone; a shared recurrent block loops a variable number of times, so one checkpoint serves different inference-compute budgets
- Pretrained on 6T tokens (first 4.5T in NVFP4), context extended to 256K, then post-trained with SFT and RLVR using a difficulty-aware loop curriculum that routes harder prompts to deeper loops
Results
- With two or more loops it outperforms larger models (Qwen3.5-9B, Gemma 4 12B, Gemma 4 26B-A4B) on average across 12 benchmarks
- With four loops it generates 1.3–2.5x fewer tokens on agentic benchmarks: extra compute goes to latent space instead of longer outputs
- Caches KV only from the last loop, keeping KV-cache footprint equal to a non-looped model
BF16, MXFP8, NVFP4 and GGUF quantized checkpoints are open-sourced.
Related event: NVIDIA Open-Sources Recurrent Model Dory1.0-4B(2 posts)→
More from Research
- Stanford Cancer Biology Best Thesis Winner Vows to Build AI Agents for Science — MengdiWang10 · 2026-11-06
- VybBench open-sourced: top image models change 90%+ of pixels on tiny edits — balajiln · 2026-11-06
- Fluency, not latency: HRI metrics offer a better benchmark for real-time AI agents — Stefania_druga · 2026-11-06
- METR: AI capability acceleration not driven by faster training compute growth — BenBajarin · 2026-11-06
- UChicago team backed by Laude and NSF builds actionable AI weather forecasts for farmers — BenBlaiszik · 2026-11-06
- Fly connectome with 138k neurons drives a robot duck's escape reflexes in sim — willknight · 2026-11-06