NVIDIA's Looped Dory1.0-4B lets you dial inference depth from 1 to 4 passes
lmoroney · x · 2026-10-20
Laurence Moroney breaks down NVIDIA Nemotron Labs' Dory1.0-4B, a looped model with an inference-time depth dial. The architecture splits layers into 8 input, a shared 20-layer recurrent block applied 1–4 times at run time, and 8 output layers, with input re-injected each loop. One checkpoint serves both quick calls (1 loop) and hard coding tasks (4 loops) — no retraining, no model swap. NVIDIA calls this elastic inference compute; the post also covers how it keeps memory flat via KV cache handling and what to benchmark before trusting it in agents.
Related event: NVIDIA Open-Sources Dory1.0-4B Looped Model with Adjustable Inference Depth(2 posts)→
More from Infra
- NeurIPS 2026 paper 'Intelligence per Watt' measures on-device AI efficiency across 1M queries — HazyResearch · 2026-10-20
- SpaceX Faces $26B Compute-Driven Earnings and a 1.3B Share Unlock in the Same Week — JOBhakdi · 2026-10-20
- Core economic flaw in the AI bull case: compute cost exceeds the benefit it delivers — SumitGup · 2026-10-20
- ijah: a new engine for running Qwen 3.8 Flash-Next locally on M5 Max Macs — SeveralViolins · 2026-10-20
- Shadeform launches six features as agent-era compute breaks existing rental models — omarsar0 · 2026-10-20
- Alex Atallah's CAP Theorem for Inference Providers: Reliability, Cost, Speed — AccBalanced · 2026-10-20